Re: [Lazarus] Problem with non-ascii chars fpc-3.0.0 vs fpc-3.1.1 under Windows

[zeljko]

On 04/28/2016 02:26 PM, Bart wrote:

Example with
set zeljko=äëï
Result of StringCodePage in sqaure brackets:

GetEnvironmentVariableUtf8: zeljko = $C3 $A4 $C3 $AB $C3 $AF [0]
GetEnvironmentVariable with UnicodeString: zeljko = $00E4 $00EB $00EF [1200]
UTF16ToUTF8 on result of previous line: zeljko = $C3 $A4 $C3 $AB $C3 $AF [65001]
GetAppConfigDirUtf8:  [65001]

As a workaround you may try to use the Unicode variant of
GetEnvironmentVariable and convert it to Utf8 with UTF16ToUtf8().
The resulting string (on my system) then has stringCodePage = CP_UTF8 again.


Thanks, that workaround works, but with 2.6.4 have another type of problem:
1.Home dir is C:\Users\Željko
2.DirectoryExistsUTF8, any file inside $HOME FileExistsUTF8 and other 
file routines from LazFileUtils works like a charm.
3.IniFiles - TMemIniFile uses UpdateFile, and of course eg. 
C:\Users\Željko\myinifile.ini fails to update because UpdateFile 
inifiles uses ForceDirectories() ...

4.I said ok, let's create it with:
 a) UTF8ToSys(AFileName) - does not work
 b) UTF8ToConsole(AFileName) - does not work
 c) UTF8ToAnsi(AFIleName) - does not work
 d) LConvEncoding.UTF8ToCp1250(AFIleName) WORKS

So, have I missed something about UTF8ToSys() and others in case of 
fpc-2.6.4 ?


NOTE: I'm talking about fpc-2.6.4 ! fpc-3.0.0 works correct in this case

zeljko


--
___
Lazarus mailing list
Lazarus@lists.lazarus.freepascal.org
http://lists.lazarus.freepascal.org/mailman/listinfo/lazarus

Re: [Lazarus] Problem with non-ascii chars fpc-3.0.0 vs fpc-3.1.1 under Windows

[Bart]

On 4/28/16, zeljko  wrote:

> This is not true under windows 10 + fpc-3.0.0 when user name is Željko.
> CurrentDir & CurrentDirUTF8 returns C:\Users\Željko\test
> GetAppConfigDirUTF8(False) returns C:\Users\Željko\test
>
> GetEnvironmentVariable &
> GetEnvironmentVariableUTF8
> returns C:\Users\L"eljko <- looks like wrong.

On Win7 (Lazarus trunk, fpc 3.0 32-bit) at least for unicode
characters inside my codepage GetEnvironmentVariableUTF8 retruns
correct string, encoded as UTF8, however StringCodePage is CP_ACP.
Not sure it this matters at all for your usecase, but GetAppConfigDir
returns a variable in UTF8 encoding where StringCodePage is CP_UTF8.

Example with
set zeljko=äëï
Result of StringCodePage in sqaure brackets:

GetEnvironmentVariableUtf8: zeljko = $C3 $A4 $C3 $AB $C3 $AF [0]
GetEnvironmentVariable with UnicodeString: zeljko = $00E4 $00EB $00EF [1200]
UTF16ToUTF8 on result of previous line: zeljko = $C3 $A4 $C3 $AB $C3 $AF [65001]
GetAppConfigDirUtf8:  [65001]

As a workaround you may try to use the Unicode variant of
GetEnvironmentVariable and convert it to Utf8 with UTF16ToUtf8().
The resulting string (on my system) then has stringCodePage = CP_UTF8 again.

Bart

--
___
Lazarus mailing list
Lazarus@lists.lazarus.freepascal.org
http://lists.lazarus.freepascal.org/mailman/listinfo/lazarus

Re: [Lazarus] Problem with non-ascii chars fpc-3.0.0 vs fpc-3.1.1 under Windows

[zeljko]

On 04/27/2016 09:00 PM, Mattias Gaertner wrote:

On Wed, 27 Apr 2016 20:30:43 +0200
zeljko  wrote:


Hi,
Just run into problem with fpc-3.0.0, OS Windows-10 64bit, 32bit
application, lazarus trunk qtlcl and win32 ws both have same problem.

fpc-3.1.1 (18.01.2016) is correct - both running from lazarus or as
standalone.

fpc-3.0.0 is not correct in both cases (even worse if running from
lazarus, then GetEnvironmentVariableUTF8 returns garbage instead of
letter Ž.

fpc-2.6.4 - works just fine.


My project doesn't use anything special - pure empty project with one
form and TMemo - no extra settings like -FcUTF8 or similar, using mode
delphi 

Problematic part is line with CACHEDIR= (see screenshoots) which is
correct with 3.1.1 and it returns correct path from this function:

function GetCacheDir_: String;
{$IFDEF MSWINDOWS}
var
s: string;
s1: string;
{$ENDIF}
begin
{$IFDEF UNIX}
Result := GetEnvironmentVariable('HOME') + PathDelim + '.holobit';
{$ELSE}
// fpc-3.1.1 returns correct values here, while fpc-3.0.0 NOT
// if environmentvariable contains non-ascii chars
s := GetEnvironmentVariable('HOMEDRIVE');
// added s := s+ to fit into mail ...
s := s + GetEnvironmentVariable('HOMEPATH')+PathDelim+'.holobit';
SetLength(s1, length(s));
OEMToAnsiBuff(@S[1], @S1[1], length(S1));


OEMToAnsiBuff expects a string in DOS codepage and returns a string in
Windows codepage.


Yes, I know that ,but that's not problem...look below...


GetEnvironmentVariableUTF8 returns an UTF-8 encoded string.

What encoding do you need for the function result?


UTF-8, but question is why 3.1.1 (and 2.6.4) returns correct result but 
3.0.0 NOT with same project and same settings.


Another thing is that project when running from IDE with fpc-3.0.0 
doesn't get correct result even for GetEnvironmentVariableUTF8(), while 
3.1.1 is correct again.




zeljko


--
___
Lazarus mailing list
Lazarus@lists.lazarus.freepascal.org
http://lists.lazarus.freepascal.org/mailman/listinfo/lazarus

Re: [Lazarus] Problem with non-ascii chars fpc-3.0.0 vs fpc-3.1.1 under Windows

[Mattias Gaertner]

On Wed, 27 Apr 2016 20:30:43 +0200
zeljko  wrote:

> Hi,
> Just run into problem with fpc-3.0.0, OS Windows-10 64bit, 32bit 
> application, lazarus trunk qtlcl and win32 ws both have same problem.
> 
> fpc-3.1.1 (18.01.2016) is correct - both running from lazarus or as 
> standalone.
> 
> fpc-3.0.0 is not correct in both cases (even worse if running from 
> lazarus, then GetEnvironmentVariableUTF8 returns garbage instead of
> letter Ž.
> 
> fpc-2.6.4 - works just fine.
> 
> 
> My project doesn't use anything special - pure empty project with one 
> form and TMemo - no extra settings like -FcUTF8 or similar, using mode 
> delphi 
> 
> Problematic part is line with CACHEDIR= (see screenshoots) which is 
> correct with 3.1.1 and it returns correct path from this function:
> 
> function GetCacheDir_: String;
> {$IFDEF MSWINDOWS}
> var
>s: string;
>s1: string;
> {$ENDIF}
> begin
>{$IFDEF UNIX}
>Result := GetEnvironmentVariable('HOME') + PathDelim + '.holobit';
>{$ELSE}
>// fpc-3.1.1 returns correct values here, while fpc-3.0.0 NOT
>// if environmentvariable contains non-ascii chars
>s := GetEnvironmentVariable('HOMEDRIVE');
>// added s := s+ to fit into mail ...
>s := s + GetEnvironmentVariable('HOMEPATH')+PathDelim+'.holobit';
>SetLength(s1, length(s));
>OEMToAnsiBuff(@S[1], @S1[1], length(S1));

OEMToAnsiBuff expects a string in DOS codepage and returns a string in
Windows codepage.
GetEnvironmentVariableUTF8 returns an UTF-8 encoded string.

What encoding do you need for the function result?

>Result := S1;
>{$ENDIF}
> end;
> 
> So, my question would be: is it known bug in 3.0.0 and will be fixed in 
> 3.0.2 or it's already merged in fixes_3_0 or I can sit and cry...or I'm 
> missing something ?

Mattias

--
___
Lazarus mailing list
Lazarus@lists.lazarus.freepascal.org
http://lists.lazarus.freepascal.org/mailman/listinfo/lazarus