Re: [Lazarus] UTF8 RTL for Windows
On 24.11.2014 03:19, luiz americo pereira camara wrote: I updated the test app to show the hexadecimal representation of the string. When {$codepage utf8} is set, all string encoding and content is right matching each other regardless of MultiByteConversionCodePage Without {$codepage utf8}: When MultiByteConversionCodePage is CP_ACP (default) one string gets the UTF8 content but code page is system ansi (1252 in my case) When MultiByteConversionCodePage is UTF8 and two strings (converted from WideString) get code page UTF8 but content is wrong Yes. $codepage is for the how the compiler parses the constants while MultiByteConversionCodePage is for the runtime behavior. In theory this is all documented at http://wiki.freepascal.org/FPC_Unicode_support#String_constants Regards, Sven -- ___ Lazarus mailing list Lazarus@lists.lazarus.freepascal.org http://lists.lazarus.freepascal.org/mailman/listinfo/lazarus
Re: [Lazarus] UTF8 RTL for Windows
On 24.11.2014 01:37, luiz americo pereira camara wrote: 2014-11-20 13:21 GMT-03:00 Mattias Gaertner mailto:nc-gaert...@netcologne.de>>: Please test and tell what you find out. Without {$codepage utf8} directive String constants will get Code Page 0 (CP_ACP) and not the 1200 (UTF16 - UnicodeString). String variables assigned to those constants will also have Code Page = 0 This is because the constant string code page is evaluated at compile time Not sure if there's a compiler command line param with same effect as {$codepage utf8} The attached program show how data loss can occur The command line parameter for this is -Fcutf8. Regards, Sven -- ___ Lazarus mailing list Lazarus@lists.lazarus.freepascal.org http://lists.lazarus.freepascal.org/mailman/listinfo/lazarus
Re: [Lazarus] UTF8 RTL for Windows
I updated the test app to show the hexadecimal representation of the string. When {$codepage utf8} is set, all string encoding and content is right matching each other regardless of MultiByteConversionCodePage Without {$codepage utf8}: When MultiByteConversionCodePage is CP_ACP (default) one string gets the UTF8 content but code page is system ansi (1252 in my case) When MultiByteConversionCodePage is UTF8 and two strings (converted from WideString) get code page UTF8 but content is wrong Luiz 2014-11-23 22:06 GMT-03:00 luiz americo pereira camara : > I added {.$codepage utf8} and all strings output as "Joao". > > Got confused. I did not to expect changes in the constant assigned to the > UnicodeString variable > > Need to check what is the correct UTF8 output: "JoA£o" or "Joao" > > Luiz > > 2014-11-23 21:37 GMT-03:00 luiz americo pereira camara > : > > >> >> 2014-11-20 13:21 GMT-03:00 Mattias Gaertner : >> >>> >>> Please test and tell what you find out. >>> >> >> >> Without {$codepage utf8} directive String constants will get Code Page 0 >> (CP_ACP) and not the 1200 (UTF16 - UnicodeString). >> >> String variables assigned to those constants will also have Code Page = 0 >> >> This is because the constant string code page is evaluated at compile time >> >> Not sure if there's a compiler command line param with same effect as >> {$codepage utf8} >> >> The attached program show how data loss can occur >> >> Luiz >> > > program testStringConstantCP; {$mode objfpc}{$H+} {.$codepage utf8} uses Classes, sysutils; function StrToHex(const S: String): String; var i: Integer; begin Result := ''; if S = '' then Exit; for i := 1 to Length(S) do begin Result := Result + IntToHex(Byte(S[i]), 0); end; end; var W: UnicodeString; S, S_2: String; SUTF8, SUTF8_2: UTF8String; begin SetMultiByteConversionCodePage(CP_UTF8); W := 'ã'; Write('W: ': 10, StringCodePage(W): 6, ' - '); WriteLn(W: 6); S := 'ã'; Write('S: ': 10,StringCodePage(S): 6, ' - '); WriteLn(S: 6, ' - ', StrToHex(S)); S_2 := W; Write('S_2: ': 10,StringCodePage(S_2): 6, ' - '); WriteLn(S_2: 6, ' - ', StrToHex(S_2)); SUTF8 := W; Write('SUTF8: ': 10,StringCodePage(SUTF8): 6, ' - '); WriteLn(SUTF8: 6, ' - ', StrToHex(SUTF8)); SUTF8_2 := S; Write('SUTF8_2: ': 10, StringCodePage(SUTF8_2): 6, ' - '); WriteLn(SUTF8_2: 6, ' - ', StrToHex(SUTF8_2)); end. -- ___ Lazarus mailing list Lazarus@lists.lazarus.freepascal.org http://lists.lazarus.freepascal.org/mailman/listinfo/lazarus
Re: [Lazarus] UTF8 RTL for Windows
I added {.$codepage utf8} and all strings output as "Joao". Got confused. I did not to expect changes in the constant assigned to the UnicodeString variable Need to check what is the correct UTF8 output: "JoA£o" or "Joao" Luiz 2014-11-23 21:37 GMT-03:00 luiz americo pereira camara : > > > 2014-11-20 13:21 GMT-03:00 Mattias Gaertner : > >> >> Please test and tell what you find out. >> > > > Without {$codepage utf8} directive String constants will get Code Page 0 > (CP_ACP) and not the 1200 (UTF16 - UnicodeString). > > String variables assigned to those constants will also have Code Page = 0 > > This is because the constant string code page is evaluated at compile time > > Not sure if there's a compiler command line param with same effect as > {$codepage utf8} > > The attached program show how data loss can occur > > Luiz > -- ___ Lazarus mailing list Lazarus@lists.lazarus.freepascal.org http://lists.lazarus.freepascal.org/mailman/listinfo/lazarus
Re: [Lazarus] UTF8 RTL for Windows
2014-11-20 13:21 GMT-03:00 Mattias Gaertner : > > Please test and tell what you find out. > Without {$codepage utf8} directive String constants will get Code Page 0 (CP_ACP) and not the 1200 (UTF16 - UnicodeString). String variables assigned to those constants will also have Code Page = 0 This is because the constant string code page is evaluated at compile time Not sure if there's a compiler command line param with same effect as {$codepage utf8} The attached program show how data loss can occur Luiz program testStringConstantCP; {$mode objfpc}{$H+} uses Classes, sysutils; var W: UnicodeString; S, S_2: String; SUTF8, SUTF8_2: UTF8String; begin SetMultiByteConversionCodePage(CP_UTF8); W := 'João'; Write('W: ': 10, StringCodePage(W), ' - '); WriteLn(W); S := 'João'; Write('S: ': 10,StringCodePage(S), ' - '); WriteLn(S); S_2 := W; Write('S_2: ': 10,StringCodePage(S_2), ' - '); WriteLn(S_2); SUTF8 := W; Write('SUTF8: ': 10,StringCodePage(SUTF8), ' - '); WriteLn(SUTF8); SUTF8_2 := S; Write('SUTF8_2: ': 10, StringCodePage(SUTF8_2), ' - '); WriteLn(SUTF8_2); end. -- ___ Lazarus mailing list Lazarus@lists.lazarus.freepascal.org http://lists.lazarus.freepascal.org/mailman/listinfo/lazarus
Re: [Lazarus] UTF8 RTL for Windows
2014-11-20 13:21 GMT-03:00 Mattias Gaertner : > > 2. The new mode: The LCL, FCL and RTL treat all "String" as UTF-8 > encoded. Most RTL file functions now work with full Unicode. > For example FileExists and aStringList.LoadFromFile(Filename) now > support full Unicode. > [..] Please test and tell what you find out. > > The FormatSettings fields are still encoded with System Code Page regardless of DefaultSystemCodePage value. While for english locales there's no problem, other locales like PT-BR have accented names in days and monthes. The problem is in windows SysUtils.GetLocaleStr function that uses non unicode Win Api function. This problem will affect also the UnicodeString RTL. Attached is a test app that shows the issue. It also has a version of GetLocaleStr that fixes the issue for the RTL (both versions) Luiz program TestUTF8FormatSettings; {$mode objfpc}{$H+} uses {$ifdef Windows} Windows, {$endif} Classes, sysutils; {$ifdef Windows} function GetLocaleStrTest(LID, LT: Longint; const Def: string): String; var L: Integer; Buf: array[0..255] of WideChar; W: WideString; begin L := GetLocaleInfoW(LID, LT, Buf, SizeOf(Buf)); if L > 0 then begin //SetString(Result, PWideChar(@Buf[0]), L - 1) leads to wrong result //Bug in Procedure SetString (Out S : AnsiString; Buf : PWideChar; Len : SizeInt) ? SetString(W, PWideChar(@Buf[0]), L - 1); Result := W; end else Result := Def; end; {$endif} var i: Integer; S: String; List: TStringList; begin WriteLn('DefaultSystemCodePage: ', DefaultSystemCodePage); DefaultSystemCodePage:=CP_UTF8; DefaultRTLFileSystemCodePage:=CP_UTF8; List := TStringList.Create; for i := 1 to 12 do begin Write(StringCodePage(DefaultFormatSettings.LongMonthNames[i]), ' - '); WriteLn(DefaultFormatSettings.LongMonthNames[i]); List.Add(DefaultFormatSettings.LongMonthNames[i]); end; for i := 1 to 7 do begin Write(StringCodePage(DefaultFormatSettings.LongDayNames[i]), ' - '); WriteLn(DefaultFormatSettings.LongDayNames[i]); List.Add(DefaultFormatSettings.LongDayNames[i]); end; {$ifdef Windows} S := GetLocaleStrTest(GetThreadLocale, LOCALE_SDAYNAME1+1, 'xx'); Write(StringCodePage(S), ' - '); WriteLn(S); List.Add(S); {$endif} List.SaveToFile('TestUTF8FormatSettingsOut.txt'); List.Destroy; end. -- ___ Lazarus mailing list Lazarus@lists.lazarus.freepascal.org http://lists.lazarus.freepascal.org/mailman/listinfo/lazarus
Re: [Lazarus] lresources.pp(3089, 67) Error: Identifier not found "RT_RCDATA"
On 11/22/2014 6:45 PM, Mattias Gaertner wrote: On Sat, 22 Nov 2014 23:27:30 +0100 Bart wrote: On 11/22/14, Joost van der Sluis wrote: Add the windows-unit to the uses section of lresources as a quick fix. Done. i pulled your update last evening and compiled... it seems to have worked... -- NOTE: No off-list assistance is given without prior approval. Please *keep mailing list traffic on the list* unless private contact is specifically requested and granted. -- ___ Lazarus mailing list Lazarus@lists.lazarus.freepascal.org http://lists.lazarus.freepascal.org/mailman/listinfo/lazarus
Re: [Lazarus] UTF8 RTL for Windows
On Sun, Nov 23, 2014 at 1:56 PM, Michael Van Canneyt wrote: > Don't worry. Computers are not scary, not really. Just look at "Terminator" > (or any other Sci-Fi involving computers), the humans always win in the > end... :-) Well, the first reports of how the unicode rtl would look like were pretty scary: Total break of the string part of millions of lines of code that people wrote with Lazarus since years. But now reading the latest report of how it will work out, i.e. that Char=WideChar only in a special mode, and that you can set some variables to get UTF-8 strings from RTL system calls, well, I haven't actually tested it yet, but it looks like that maybe our code will not break and maybe we won't need to review/fix hundreds of thousands of lines of code that have worked since years -- Felipe Monteiro de Carvalho -- ___ Lazarus mailing list Lazarus@lists.lazarus.freepascal.org http://lists.lazarus.freepascal.org/mailman/listinfo/lazarus
[Lazarus] ObjectInspector FCurrentEdit.Invalidate
Please review attached patch. - ChangeBounds calls Invalidate itself. Please apply if OK, thanx. Sandro Index: components/ideintf/objectinspector.pp === --- components/ideintf/objectinspector.pp (revisione 46974) +++ components/ideintf/objectinspector.pp (copia locale) @@ -2631,10 +2631,7 @@ end; //debugln('TOICustomPropertyGrid.AlignEditComponents A ',dbgsName(FCurrentEdit),' ',dbgs(EditCompRect)); if not CompareTopLeft(FCurrentEdit.BoundsRect,EditCompRect) then - begin FCurrentEdit.BoundsRect:=EditCompRect; -FCurrentEdit.Invalidate; - end; end; end; end; -- ___ Lazarus mailing list Lazarus@lists.lazarus.freepascal.org http://lists.lazarus.freepascal.org/mailman/listinfo/lazarus
Re: [Lazarus] ObjectInspector horizontal resize bug
On Sun, Nov 23, 2014 at 8:18 PM, Sandro Cumerlato wrote: > Please review attached patch. > > - selected edit control doesn't resize properly after a horizontal resize. Applied, thanks. Juha -- ___ Lazarus mailing list Lazarus@lists.lazarus.freepascal.org http://lists.lazarus.freepascal.org/mailman/listinfo/lazarus
Re: [Lazarus] ObjectInspector SpeedButton background issue
On Sun, Nov 23, 2014 at 7:50 PM, Sandro Cumerlato wrote: > Please review attached patch. > > - removed code that prevent SpeedButton background erase. Applied, thanks. Juha -- ___ Lazarus mailing list Lazarus@lists.lazarus.freepascal.org http://lists.lazarus.freepascal.org/mailman/listinfo/lazarus
[Lazarus] ObjectInspector horizontal resize bug
Please review attached patch. - selected edit control doesn't resize properly after a horizontal resize. Apply if OK, thanx. Sandro Index: components/ideintf/objectinspector.pp === --- components/ideintf/objectinspector.pp (revisione 46974) +++ components/ideintf/objectinspector.pp (copia locale) @@ -2586,11 +2586,6 @@ and (r1.Right=r2.Right) and (r1.Bottom=r2.Bottom); end; - function CompareTopLeft(r1,r2:TRect):boolean; - begin -Result := (r1.Left=r2.Left) and (r1.Top=r2.Top); - end; - // AlignEditComponents begin if ItemIndex>=0 then @@ -2630,7 +2625,7 @@ Inc(EditCompRect.Bottom); end; //debugln('TOICustomPropertyGrid.AlignEditComponents A ',dbgsName(FCurrentEdit),' ',dbgs(EditCompRect)); - if not CompareTopLeft(FCurrentEdit.BoundsRect,EditCompRect) then + if not CompareRectangles(FCurrentEdit.BoundsRect,EditCompRect) then begin FCurrentEdit.BoundsRect:=EditCompRect; FCurrentEdit.Invalidate; -- ___ Lazarus mailing list Lazarus@lists.lazarus.freepascal.org http://lists.lazarus.freepascal.org/mailman/listinfo/lazarus
[Lazarus] ObjectInspector SpeedButton background issue
Please review attached patch. - removed code that prevent SpeedButton background erase. Sandro Index: components/ideintf/objectinspector.pp === --- components/ideintf/objectinspector.pp (revisione 46974) +++ components/ideintf/objectinspector.pp (copia locale) @@ -2660,11 +2660,6 @@ ValueRect := FullRect; Inc(FullRect.Bottom, FRowSpacing); - if ARow = FItemIndex then -if Layout = oilHorizontal then - if Assigned(FCurrentButton) and (FCurrentButton.Visible) then -Dec(FullRect.Right, FCurrentButton.Width); - if Layout = oilHorizontal then begin NameRect.Right:=SplitterX; -- ___ Lazarus mailing list Lazarus@lists.lazarus.freepascal.org http://lists.lazarus.freepascal.org/mailman/listinfo/lazarus
Re: [Lazarus] ObjectInspector DefaultItemHeight issue
On Sunday, November 23, 2014, Sandro Cumerlato wrote: > Please review attached patch: > > - uniform DefaultItemHeight to 22 pixels; > - avoid DefaultItemHeight < ComboBox Height (- 3). > > Apply if OK, thanx. > Applied the modified version from personal chat. Thanks. Juha -- ___ Lazarus mailing list Lazarus@lists.lazarus.freepascal.org http://lists.lazarus.freepascal.org/mailman/listinfo/lazarus
[Lazarus] ObjectInspector DefaultItemHeight issue
Please review attached patch: - uniform DefaultItemHeight to 22 pixels; - avoid DefaultItemHeight < ComboBox Height (- 3). Apply if OK, thanx. Sandro Index: components/ideintf/objectinspector.pp === --- components/ideintf/objectinspector.pp (revisione 46970) +++ components/ideintf/objectinspector.pp (copia locale) @@ -474,7 +474,7 @@ property CurrentEditValue: string read GetCurrentEditValue write SetCurrentEditValue; property DefaultItemHeight:integer read FDefaultItemHeight - write FDefaultItemHeight default 25; + write FDefaultItemHeight default 22; property DrawHorzGridLines: Boolean read FDrawHorzGridLines write SetDrawHorzGridLines default True; property ExpandedProperties: TStringList read FExpandedProperties @@ -994,10 +994,8 @@ FHintManager := THintWindowManager.Create; FActiveRowBmp := CreateBitmapFromResourceName(HInstance, 'pg_active_row'); - if DefItemHeight<3 then -FDefaultItemHeight:=ValueComboBox.Height-3 - else -FDefaultItemHeight:=DefItemHeight; + if DefItemHeight-- ___ Lazarus mailing list Lazarus@lists.lazarus.freepascal.org http://lists.lazarus.freepascal.org/mailman/listinfo/lazarus
Re: [Lazarus] UTF8 RTL for Windows
On 2014-11-23 12:56, Michael Van Canneyt wrote: > the humans always win in the end... :-) ROFL > Phew... At least something we did better in the whole string mess ... ;) 9/10 times FPC does everything better than Delphi. Regards, - Graeme - -- fpGUI Toolkit - a cross-platform GUI toolkit using Free Pascal http://fpgui.sourceforge.net/ -- ___ Lazarus mailing list Lazarus@lists.lazarus.freepascal.org http://lists.lazarus.freepascal.org/mailman/listinfo/lazarus
[Lazarus] Component standard Height (WidgetSet dependant)
Hello, is there another way to retrieve the component standard Height (WidgetSet dependant) at runtime? CB := TComboBox.Create(self); h := CB.Height; CB.Free; CB := nil; Thanx in advance! Sandro -- ___ Lazarus mailing list Lazarus@lists.lazarus.freepascal.org http://lists.lazarus.freepascal.org/mailman/listinfo/lazarus
Re: [Lazarus] UTF8 RTL for Windows
On Sun, 23 Nov 2014 13:56:42 +0100 (CET) Michael Van Canneyt wrote: >[...] > Anyway, I was just trying to say that a 1-byte string is not necessarily > UTF-8 in FPC 2.7.1. Yes, you can still store anything you like in strings. And you can store UTF-8 in a string and say it is not. Mattias -- ___ Lazarus mailing list Lazarus@lists.lazarus.freepascal.org http://lists.lazarus.freepascal.org/mailman/listinfo/lazarus
Re: [Lazarus] OI Checkboxes
On Sun, Nov 23, 2014 at 4:43 PM, Sandro Cumerlato wrote: > I cannot test the QT widgetset, sorry, but I would suggest attached patch > instead. Applied, thanks. Juha -- ___ Lazarus mailing list Lazarus@lists.lazarus.freepascal.org http://lists.lazarus.freepascal.org/mailman/listinfo/lazarus
Re: [Lazarus] OI Checkboxes
On 22 November 2014 at 19:45, Juha Manninen wrote: > On Fri, Nov 21, 2014 at 4:31 PM, FreeMan > wrote: > > Thank you, uncommenting resolve my problem. > > Without those lines indeed Lazarus built with QT crashes when Color > combobox list is opened. > Why does it happen? > I returned the code in r46966. > > Juha > > I cannot test the QT widgetset, sorry, but I would suggest attached patch instead. Sandro Index: components/ideintf/objectinspector.pp === --- components/ideintf/objectinspector.pp (revisione 46968) +++ components/ideintf/objectinspector.pp (copia locale) @@ -942,6 +942,7 @@ AutoSize:=false; SetBounds(0,-30,Width,Height); // hidden DropDownCount:=20; +ItemHeight:=17; Parent:=Self; OnMouseDown := @ValueControlMouseDown; OnMouseMove := @ValueControlMouseMove; @@ -2632,11 +2633,6 @@ if not CompareTopLeft(FCurrentEdit.BoundsRect,EditCompRect) then begin FCurrentEdit.BoundsRect:=EditCompRect; -// If ItemHeight is not set, Lazarus built with QT crashes -// when Color combobox list is opened. Why is that? -if FCurrentEdit is TComboBox then - TComboBox(FCurrentEdit).ItemHeight:=EditCompRect.Bottom-EditCompRect.Top-6; -// FCurrentEdit.Invalidate; end; end; -- ___ Lazarus mailing list Lazarus@lists.lazarus.freepascal.org http://lists.lazarus.freepascal.org/mailman/listinfo/lazarus
Re: [Lazarus] CodeTools error when using 'section' keyword
Works fine now, thank you very much for the fix! Michael Am 20.11.14 um 15:01 schrieb Mattias Gaertner: On Thu, 23 Oct 2014 21:57:28 +0200 Michael Ring wrote: [...] This is my test program: program hello; const DEVCFG3_DEFAULT=1; const devcfg3: longWord = DEVCFG3_DEFAULT; section '.devcfg3'; devcfg2: longWord = DEVCFG3_DEFAULT; section '.devcfg2'; Fixed. Please test. Mattias -- ___ Lazarus mailing list Lazarus@lists.lazarus.freepascal.org http://lists.lazarus.freepascal.org/mailman/listinfo/lazarus -- ___ Lazarus mailing list Lazarus@lists.lazarus.freepascal.org http://lists.lazarus.freepascal.org/mailman/listinfo/lazarus
Re: [Lazarus] UTF8 RTL for Windows
On Sun, 23 Nov 2014, Mattias Gaertner wrote: True. Although many programmers misunderstand what this means. It is not as scary as it sounds. To all the scared people: Don't worry. Computers are not scary, not really. Just look at "Terminator" (or any other Sci-Fi involving computers), the humans always win in the end... :-) Additionally, most basic File I/O routines now correctly call the underlying OS-es file routines with the codepage the OS expects (which is WideString on Windows). Is it safe to say UTF-16? Or are there still UCS-2 Windows? I think some older versions of Windows are still UCS2, but I believe as of Windows 2000, it is all UTF-16. However, I am not an expert. The exact behaviour of the RTL is controlled by a couple of variables: DefaultSystemCodePage, DefaultFileSystemCodePage , DefaultRTLFileSystemCodePage. Yes, that's the important bit that FPC made better than Delphi. :) Phew... At least something we did better in the whole string mess ... ;) Anyway, I was just trying to say that a 1-byte string is not necessarily UTF-8 in FPC 2.7.1. Michael. -- ___ Lazarus mailing list Lazarus@lists.lazarus.freepascal.org http://lists.lazarus.freepascal.org/mailman/listinfo/lazarus
Re: [Lazarus] lresources.pp(3089, 67) Error: Identifier not found "RT_RCDATA"
Am 22.11.2014 22:28 schrieb "Joost van der Sluis" : > > On 11/22/2014 08:30 PM, waldo kitty wrote: >> >> >> fpc trunk r29110 >> lazarus trunk r46966 >> vista 32-bit >> >> i just updated fpc trunk and lazarus trunk... fpc compiled ok... lazarus >> fails with >> >> lresources.pp(3089,67) Error: Identifier not found "RT_RCDATA" > > > Add the windows-unit to the uses section of lresources as a quick fix. Would it he better if for consistency we export the RT_* constants from System as well on Windows platforms? Regards, Sven -- ___ Lazarus mailing list Lazarus@lists.lazarus.freepascal.org http://lists.lazarus.freepascal.org/mailman/listinfo/lazarus
Re: [Lazarus] UTF8 RTL for Windows
Am 23.11.2014 00:15 schrieb "Mattias Gaertner" : > > Additionally, most basic File I/O routines now correctly call the underlying > > OS-es file routines with the codepage the OS expects (which is WideString on Windows). > > Is it safe to say UTF-16? Or are there still UCS-2 Windows? Till NT 4 inclusive it's UCS-2, since Windows 2000 it's UTF-16 (I don't know and especially don't care about 9x). Regards, Sven -- ___ Lazarus mailing list Lazarus@lists.lazarus.freepascal.org http://lists.lazarus.freepascal.org/mailman/listinfo/lazarus