Re: [fpc-devel] fpc compiler wrong encoding in console output on Windows

[Ondrej Pokorny via fpc-devel]

On 31.08.2023 17:54, Tomas Hajny via fpc-devel wrote:

On 2023-08-31 17:08, Ondrej Pokorny via fpc-devel wrote:

:) We are not at the finish yet. Positive outcome will be when it gets
fixed in the compiler source.


Yes, I know. I checked your patch briefly now and I don't like the 
fact that it basically duplicates the code of the two original 
TScannerFile methods. This isn't very good from maintainability point 
of view, because if somebody changes/fixes some issue in one, (s)he 
may easily forget about the other. I see several potential solutions:


2) Always use the ansistring version when parsing the comments. 
Obviously, this increases compilation time for directives parsing 
somewhat - we could possibly measure the impact and then decide if the 
increase is important.


That is actually what I started with. Then I saw that there are some 
parts of the code that demand ShortString output and decided to keep 
both versions.


But basically it is possible to have the 2 ShortString variants use the 
new Long variants + check the Length and valid characters (to me it 
looks like there really should be checks for ASCII characters when 
ShortString is involved). That would be the simplest way to get rid of 
doubled code. I don't think the compiler performance will be affected 
that much and AFAIK maintainability is more important than performance 
anyway.


Or the bigger step would be to make the compiler unicode-ready:

var
  ä: string;

+ file names, paths etc.

Then only the new Long variant would be needed :)

Ondrej

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] fpc compiler wrong encoding in console output on Windows

[Ondrej Pokorny via fpc-devel]

Am 31.08.2023 um 9:59 schrieb Tomas Hajny via fpc-devel:

On 2023-08-30 23:38, Ondrej Pokorny via fpc-devel wrote:

Am 30.08.2023 um 20:21 schrieb Ondrej Pokorny via fpc-devel:
Having seen the outputs, I think that the compiler just ignores the 
source file encoding for {$MESSAGE} and {$NOTE}. It reads them 
always as ANSI and then converts them to DOS-whatever.


That would explain why UTF-8 byte stream is encoded into DOS CP.

So the fix should be quite easy - when {$MESSAGE} or {$NOTE} is read 
into a string, set the correct codepage of the string.


I was correct in my assumption and I was able to fix it:
https://gitlab.com/freepascal.org/fpc/source/-/merge_requests/482

On the other hand, when I read the $CODEPAGE docs:
https://www.freepascal.org/docs-html/prog/progsu87.html#x95-940001.3.4
There it is stated that only literal strings follow $CODEPAGE and the
actual code must be in US-ASCII.

But you know: Delphi compatibility :) ...and there is no "illegal
character" compiler error as it is for:

var
  ä: string;

so one would expect {$note ä} to show up correctly.


Indeed. I'm glad that our discussion (although triggered by my mistake 
:-( ) led to a positive outcome at last. :-)


:) We are not at the finish yet. Positive outcome will be when it gets 
fixed in the compiler source.


Ondrej

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] fpc compiler wrong encoding in console output on Windows

[Ondrej Pokorny via fpc-devel]

Am 30.08.2023 um 20:21 schrieb Ondrej Pokorny via fpc-devel:

On 30.08.2023 17:35, Tomas Hajny via fpc-devel wrote:

On 2023-08-30 17:23, Ondrej Pokorny via fpc-devel wrote:

Sorry to bother you with something as trivial: is your t2.pas file
really encoded in UTF-8?

Because if I compile an ANSI file with the {$codepage utf8}
declaration, then I get "correct" output. But obviously this is very
wrong.

You can try yourself with the attached files. So maybe this is your 
mistake?


Well, you're right, this was indeed my mistake, shame on me. :-( Then 
I can confirm that the compiler behaviour is indeed wrong (although I 
have no clue why it behaves that way).


Having seen the outputs, I think that the compiler just ignores the 
source file encoding for {$MESSAGE} and {$NOTE}. It reads them always 
as ANSI and then converts them to DOS-whatever.


That would explain why UTF-8 byte stream is encoded into DOS CP.

So the fix should be quite easy - when {$MESSAGE} or {$NOTE} is read 
into a string, set the correct codepage of the string.


I was correct in my assumption and I was able to fix it: 
https://gitlab.com/freepascal.org/fpc/source/-/merge_requests/482


On the other hand, when I read the $CODEPAGE docs: 
https://www.freepascal.org/docs-html/prog/progsu87.html#x95-940001.3.4
There it is stated that only literal strings follow $CODEPAGE and the 
actual code must be in US-ASCII.


But you know: Delphi compatibility :) ...and there is no "illegal 
character" compiler error as it is for:


var
  ä: string;

so one would expect {$note ä} to show up correctly.

Ondrej

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] fpc compiler wrong encoding in console output on Windows

[Ondrej Pokorny via fpc-devel]

On 30.08.2023 17:35, Tomas Hajny via fpc-devel wrote:

On 2023-08-30 17:23, Ondrej Pokorny via fpc-devel wrote:

Sorry to bother you with something as trivial: is your t2.pas file
really encoded in UTF-8?

Because if I compile an ANSI file with the {$codepage utf8}
declaration, then I get "correct" output. But obviously this is very
wrong.

You can try yourself with the attached files. So maybe this is your 
mistake?


Well, you're right, this was indeed my mistake, shame on me. :-( Then 
I can confirm that the compiler behaviour is indeed wrong (although I 
have no clue why it behaves that way).


Having seen the outputs, I think that the compiler just ignores the 
source file encoding for {$MESSAGE} and {$NOTE}. It reads them always as 
ANSI and then converts them to DOS-whatever.


That would explain why UTF-8 byte stream is encoded into DOS CP.

So the fix should be quite easy - when {$MESSAGE} or {$NOTE} is read 
into a string, set the correct codepage of the string.


Ondrej

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] fpc compiler wrong encoding in console output on Windows

[Ondrej Pokorny via fpc-devel]

On 30.08.2023 17:04, Tomas Hajny via fpc-devel wrote:

On 2023-08-30 16:32, Ondrej Pokorny via fpc-devel wrote:

If you mean
https://stackoverflow.com/questions/56419639/what-does-beta-use-unicode-utf-8-for-worldwide-language-support-actually-do, 


then no, I don't have this option checked. However, even if I had it
checked (I don't), it couldn't make a difference, because I didn't
rely on the output displayed on console, but I actually checked the
codepage of the compiler output by redirecting it to a file and
checking the content of that file as well.

So did I:

See output.txt in the attachment. It is the output from redirection
with "> output.txt". The output is double-encoded on my system (UTF-8
as CP-850) - it means I have to convert it first from CP-850 to ANSI,
then I see correct raw UTF-8 byte stream.


Interesting, but it doesn't change anything on the fact that my result 
is different (and correct) and I don't think that my configuration is 
special in any way...


Sorry to bother you with something as trivial: is your t2.pas file 
really encoded in UTF-8?


Because if I compile an ANSI file with the {$codepage utf8} declaration, 
then I get "correct" output. But obviously this is very wrong.


You can try yourself with the attached files. So maybe this is your mistake?

Ondrej
{$codepage utf8}

program TestNote;

uses
 Windows;
begin
 WriteLn (GetACP);
 WriteLn (GetConsoleOutputCP);
 {$note ä}
end.

{$codepage utf8}

program TestNote;

uses
 Windows;
begin
 WriteLn (GetACP);
 WriteLn (GetConsoleOutputCP);
 {$note ä}
end.

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] fpc compiler wrong encoding in console output on Windows

[Ondrej Pokorny via fpc-devel]

On 30.08.2023 17:04, Tomas Hajny via fpc-devel wrote:

On 2023-08-30 16:32, Ondrej Pokorny via fpc-devel wrote:

If you mean
https://stackoverflow.com/questions/56419639/what-does-beta-use-unicode-utf-8-for-worldwide-language-support-actually-do, 


then no, I don't have this option checked. However, even if I had it
checked (I don't), it couldn't make a difference, because I didn't
rely on the output displayed on console, but I actually checked the
codepage of the compiler output by redirecting it to a file and
checking the content of that file as well.

So did I:

See output.txt in the attachment. It is the output from redirection
with "> output.txt". The output is double-encoded on my system (UTF-8
as CP-850) - it means I have to convert it first from CP-850 to ANSI,
then I see correct raw UTF-8 byte stream.


Interesting, but it doesn't change anything on the fact that my result 
is different (and correct) and I don't think that my configuration is 
special in any way...


Yes, it is strange. Mattias and Zeljko have the same output like me 
(Zeljko wrote about it in the Lazarusdev mailing list). So, currently 
you are the only one with different result from a group of 4 people 
involved in the same test :)



Could you please change my test program sent earlier by changing 
"WriteLn (GetConsoleCP);" to "WriteLn (GetConsoleOutputCP);" recompile 
and check output of that one (GetConsoleCP is not useful in this 
context, that was my fault :-( ) - just in case?


GetConsoleOutputCP = GetConsoleCP = 850 on my system.


Do you get a different behaviour if using a stock 3.2.2 compiler (the 
one distributed by the FPC team, not a Lazarus distribution)?


Stock FPC 3.2.2 has the same output:

Free Pascal Compiler version 3.2.2 [2021/05/15] for i386
Copyright (c) 1993-2021 by Florian Klaempfl and others
Target OS: Win32 for i386
Compiling TestNote.lpr
TestNote.lpr(11,3) Note: User defined: ä
Linking TestNote.exe
13 lines compiled, 0.1 sec, 28864 bytes code, 1332 bytes data
1 note(s) issued

Ondrej

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] fpc compiler wrong encoding in console output on Windows

[Ondrej Pokorny via fpc-devel]

On 30.08.2023 16:23, Tomas Hajny via fpc-devel wrote:

On 2023-08-30 15:26, Ondrej Pokorny via fpc-devel wrote:

On 30.08.2023 13:49, Mattias Gaertner via fpc-devel wrote:

On 30.08.23 12:04, Tomas Hajny via fpc-devel wrote:

On 2023-08-28 15:53, Ondrej Pokorny via fpc-devel wrote:
[...]
Your output is different from mine (using a freshly compiled trunk 
compiler):

Free Pascal Compiler version 3.3.1 [2023/08/29] for i386
Copyright (c) 1993-2023 by Florian Klaempfl and others
Target OS: Win32 for i386
Compiling t2.pas
t2.pas(4,2) Note: User defined: ä


Then you don't have default windows setup. Maybe you installed the 
cp 65001?


Yes, Tomas, in other words: maybe you enabled "Beta support for UTF-8"
in language options on your Windows?


If you mean 
https://stackoverflow.com/questions/56419639/what-does-beta-use-unicode-utf-8-for-worldwide-language-support-actually-do, 
then no, I don't have this option checked. However, even if I had it 
checked (I don't), it couldn't make a difference, because I didn't 
rely on the output displayed on console, but I actually checked the 
codepage of the compiler output by redirecting it to a file and 
checking the content of that file as well.


So did I:

See output.txt in the attachment. It is the output from redirection with 
"> output.txt".


The output is double-encoded on my system (UTF-8 as CP-850) - it means I 
have to convert it first from CP-850 to ANSI, then I see correct raw 
UTF-8 byte stream.


Ondrej
Free Pascal Compiler version 3.3.1 [2023/08/21] for i386
Copyright (c) 1993-2023 by Florian Klaempfl and others
Target OS: Win32 for i386
Compiling TestNote.lpr
TestNote.lpr(11,3) Note: User defined: اد
Linking TestNote.exe
13 lines compiled, 0.1 sec, 39232 bytes code, 1412 bytes data
1 note(s) issued
___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] fpc compiler wrong encoding in console output on Windows

[Ondrej Pokorny via fpc-devel]

On 30.08.2023 13:49, Mattias Gaertner via fpc-devel wrote:

On 30.08.23 12:04, Tomas Hajny via fpc-devel wrote:

On 2023-08-28 15:53, Ondrej Pokorny via fpc-devel wrote:
[...]
Your output is different from mine (using a freshly compiled trunk 
compiler):

Free Pascal Compiler version 3.3.1 [2023/08/29] for i386
Copyright (c) 1993-2023 by Florian Klaempfl and others
Target OS: Win32 for i386
Compiling t2.pas
t2.pas(4,2) Note: User defined: ä


Then you don't have default windows setup. Maybe you installed the cp 
65001?


Yes, Tomas, in other words: maybe you enabled "Beta support for UTF-8" 
in language options on your Windows?




[...]
However, I run the compiler from regular command line, not from Lazarus. 


I get the same in cmd.exe and via TProcess.


Agreed.


[...] What do you get when running the following test program (you 
can simulate the same conditions e.g. by naming the resulting binary 
fpc.exe and configuring Lazarus to use this "compiler"):


uses
  Windows;
begin
  WriteLn (GetACP);
  WriteLn (GetConsoleCP);


ACP is 1252
ConsoleCP 437


On my system:

ACP is 1252
ConsoleCP is 850

(437 is DOS-US, 850 is DOS-Latin-1)

https://en.wikipedia.org/wiki/Code_page_437
https://en.wikipedia.org/wiki/Code_page_850

Ondrej

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

[fpc-devel] fpc compiler wrong encoding in console output on Windows

[Ondrej Pokorny via fpc-devel]

Hello,

when compiling a program with fpc on Windows, fpc seems to have the 
console output encoding wrong.


Example program (I am using "ä" because I am on german OS, so "ä" is in 
my DOS Latin and also in my ANSI codepage):


{$codepage utf8}
program TestNote;
begin
{$note ä}
end.

The console output is:

Free Pascal Compiler version 3.3.1 [2023/08/21] for i386
Copyright (c) 1993-2023 by Florian Klaempfl and others
(1002) Target OS: Win32 for i386
(3104) Compiling TestNote.lpr
p:\programs\_zk\TestNote\TestNote.lpr(6,2) Note: (2025) User defined: ä
(9015) Linking P:\programs\_zk\TestNote\lib\Debug\i386-win32\TestNote.exe
(1008) 8 lines compiled, 0.1 sec, 78736 bytes code, 2148 bytes data
(1022) 2 hint(s) issued
(1023) 1 note(s) issued

So it looks like the FPC compiler output is raw utf-8 instead of DOS 
Latin codepage. Is this wanted?


If yes, then Lazarus could take advantage of it and read the console 
output from FPC as UTF-8. Currently wrong characters are shown in 
Lazarus messages window on Windows.


If not and console (DOS Latin) codepage should be used by default, would 
it make sense to add a compiler command line option to set the output 
codepage? In that case Lazarus could tell FPC to use UTF-8.


Ondrej

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] Memory leak: bug ot as designed?

[Ondrej Pokorny via fpc-devel]

Nice one :) I am also interested in the answer. IMO it is a bug because 
the code is valid - https://www.freepascal.org/docs-html/ref/refse10.html


FYI: Delphi doesn't create a memory leak.

Ondrej

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] Unexpected "Range check error while evaluating constants" when compiling for Win64

[Ondrej Pokorny via fpc-devel]

On 12.02.2023 23:08, Bart via fpc-devel wrote:

On Sun, Feb 12, 2023 at 10:47 PM Jonas Maebe via fpc-devel
 wrote:


Afaik it's because "default" doesn't support qword values, iirc because
of (at least old) Delphi compatibility.

OK, I removed the "default".

Theoretically would that lead to problems when storing the value in
lfm and reading it back using different bitness versions of Lazarus?
Should I even publish such a property?
(I don't think Lazarus has a property editor for HKEY...)


I wouldn't publish the HKEY property but create an own enumeration for it:

TRegKey = (rgClassesRoot, rgCurrentUser, )

and publish this instead. Pro: you can put only valid values into the 
property and object inspector shows you nice enum names instead of 
strange integers.


Then make a simple transformation to HKEY:

RegKeyToHKey: array[TRegKey] of HKEY = (HKEY_CLASSES_ROOT, 
HKEY_CURRENT_USER, ...);


Ondrej

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] Question on constref

[Ondrej Pokorny via fpc-devel]

On 02.02.2023 14:22, Sven Barth via fpc-devel wrote:
Only because you can't see a reason for it doesn't mean that it 
doesn't exist.


That is exactly why I asked :)

Ondrej

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] Question on constref

[Ondrej Pokorny via fpc-devel]

On 02.02.2023 12:19, Marco van de Voort via fpc-devel wrote:

On 2-2-2023 12:00, Ondrej Pokorny via fpc-devel wrote:
The only disadvantage is that you get a FreeAndNil copy for every 
type you pass into the parameter (?)
Are they actually made global ? Will two freeandnil in 
different units use the same?


But that is just curiousity, IMHO this remedy is worse than problem. 
Obviously even.


In case FreeAndNil is inlined, there is no problem, is there?

generic procedure FreeAndNil(var Obj: T); inline;
var
  Temp: TObject;
begin
  Temp := Obj;
  Obj := nil;
  Temp.Free;
end;

I checked now and on i386-win32 the generic FreeAndNil is actually inlined:

program test;

{$mode objfpc}{$h+}
{$modeswitch ImplicitFunctionSpecialization}

generic procedure FreeAndNil(var Obj: T);
var
  Temp: TObject;
begin
  Temp := Obj;
  Obj := nil;
  Temp.Free;
end;

type
  TSomeObject = class(TObject);
  TSomeObject2 = class(TObject);

var
  X: TSomeObject;
  X2: TSomeObject2;
const
  IsNil: array[Boolean] of string = ('nil', 'assigned');
begin
  X := TSomeObject.Create;
  X2 := TSomeObject2.Create;
  FreeAndNil(X);
  FreeAndNil(X2);
  Writeln('X  ', IsNil[Assigned(X)]);
  Writeln('X2 ', IsNil[Assigned(X2)]);

  ReadLn;
end.

So everything good!

Ondrej

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] Question on constref

[Ondrej Pokorny via fpc-devel]

On 02.02.2023 12:25, Mattias Gaertner via fpc-devel wrote:

On Thu, 2 Feb 2023 12:00:31 +0100
Ondrej Pokorny via fpc-devel  wrote:


[...]
Delphi cannot do it. FPC rocks!

+1


The only disadvantage is that you get a FreeAndNil copy for every
type you pass into the parameter (?)

Do we want to change it in SysUtils? :)

I got a feeling this solves a problem no one had.


I agree with that.

But you know, pascal is strongly typed. You should get a compiler 
warning when passing an incompatible parameter and not an AV in runtime. 
Like the Pos(Variant) issue :)


Ondrej

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] Question on constref

[Ondrej Pokorny via fpc-devel]

On 02.02.2023 11:00, Michael Van Canneyt via fpc-devel wrote:

In userspace, the best seems

Function FreeAndNil(Obj : T) : T;

begin
  Obj.Free;
  Result:=Nil;
end;

With automatic type inference for generics, this allows you to do

A:=FreeAndNil(A);


Michael, bringing your idea to the next step, FPC can do it !!!

program test;

{$mode objfpc}{$h+}
{$modeswitch ImplicitFunctionSpecialization}

generic procedure FreeAndNil(var Obj: T);
var
  Temp: TObject;
begin
  Temp := Obj;
  Obj := nil;
  Temp.Free;
end;

type
  TSomeObject = class(TObject);

var
  X: TSomeObject;
begin
  X := TSomeObject.Create;
  FreeAndNil(X);
  if Assigned(X) then
    Writeln('assigned')
  else
    Writeln('nil');
  ReadLn;
end.

-

Delphi cannot do it. FPC rocks!

The only disadvantage is that you get a FreeAndNil copy for every type 
you pass into the parameter (?)


Do we want to change it in SysUtils? :)

Ondrej

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] Question on constref

[Ondrej Pokorny via fpc-devel]

On 02.02.2023 10:22, Michael Van Canneyt via fpc-devel wrote:

On Thu, 2 Feb 2023, Ondrej Pokorny via fpc-devel wrote:

On 02.02.2023 10:15, Michael Van Canneyt via fpc-devel wrote:

On Thu, 2 Feb 2023, Ondrej Pokorny via fpc-devel wrote:
I myself cannot think of any real use case of constref other than 
hacks like the FreeAndNil in recent Delphi versions:


procedure FreeAndNil(const [ref] Obj: TObject);

that needs the ref because it changes it to nil (so is not really a 
const - it used to be an untyped var). As a result FreeAndNil 
allows you to happily pass read-only properties to it that are then 
nilled.


Which is a clear violation of the concept of Const... :/


Exactly.


Probably an ill-advised "fix" to the problem that FreeAndNil accepted 
an untyped var

and you could basically pass anything.


Yes. Solving one problem by creating a different one. The best would be 
if the FreeAndNil declaration alternated between constref and untyped 
var between builds. In that case you would be able to hunt both the 
issues :)


Seriously, having a [loose] modifier would be much more useful, IMO:

FreeAndNil(var [loose] Obj: TObject);

That would allow you to pass any TObject descendant.

Ondrej

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] Question on constref

[Ondrej Pokorny via fpc-devel]

On 02.02.2023 10:15, Michael Van Canneyt via fpc-devel wrote:

On Thu, 2 Feb 2023, Ondrej Pokorny via fpc-devel wrote:
I myself cannot think of any real use case of constref other than 
hacks like the FreeAndNil in recent Delphi versions:


procedure FreeAndNil(const [ref] Obj: TObject);

that needs the ref because it changes it to nil (so is not really a 
const - it used to be an untyped var). As a result FreeAndNil allows 
you to happily pass read-only properties to it that are then nilled.


Which is a clear violation of the concept of Const... :/


Exactly.

Ondrej

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] Question on constref

[Ondrej Pokorny via fpc-devel]

On 02.02.2023 07:42, Sven Barth via fpc-devel wrote:
The case when you *need* a constant reference. Case in point: the 
passing of TGuid in IInterface.QueryInterface. Delphi code relies on 
it being a reference, but “const” does not guarantee that for all 
platforms.


Maybe I am missing something, could you please explain why 
IInterface.QueryInterface needs constref?


Delphi definition doesn't use it (Delphi 11):

  IInterface = interface
    ['{---C000-0046}']
    function QueryInterface(const IID: TGUID; out Obj): HResult; stdcall;
    function _AddRef: Integer; stdcall;
    function _Release: Integer; stdcall;
  end;

FPC uses it in the definition:

   IUnknown = interface
 ['{---C000-0046}']
 function QueryInterface({$IFDEF 
FPC_HAS_CONSTREF}constref{$ELSE}const{$ENDIF} iid : tguid;out obj) : 
longint;{$IFNDEF WINDOWS}cdecl{$ELSE}stdcall{$ENDIF};
 function _AddRef : longint;{$IFNDEF 
WINDOWS}cdecl{$ELSE}stdcall{$ENDIF};
 function _Release : longint;{$IFNDEF 
WINDOWS}cdecl{$ELSE}stdcall{$ENDIF};

   end;

but when I check all the implementations of QueryInterface in objpas.inc 
I see that the iid is paased to TObject.QueryInterface


  function TObject.GetInterface(const iid : tguid;out obj) : boolean;

that has a const without ref.

---

I myself cannot think of any real use case of constref other than hacks 
like the FreeAndNil in recent Delphi versions:


procedure FreeAndNil(const [ref] Obj: TObject);

that needs the ref because it changes it to nil (so is not really a 
const - it used to be an untyped var). As a result FreeAndNil allows you 
to happily pass read-only properties to it that are then nilled.


Maybe somebody knows a real use case?

Ondrej

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] Function to find if IP address is local

[Ondrej Pokorny via fpc-devel]

On 09.01.2023 09:08, Michael Van Canneyt wrote:

On Mon, 9 Jan 2023, Ondrej Pokorny via fpc-devel wrote:
Is there a routine in RTL to find out if a given IPv4 address is from 
the private address range (10.* etc)?


No, not to my knowledge. Feel free to add it somewhere in fcl-net.


Here it is:

function NetAddrIsPrivate(const IP: in_addr): Boolean;
begin
  Result :=
    // 10.0.0.0 – 10.255.255.255
    (IP.s_bytes[1]=10)
    // 172.16.0.0 – 172.31.255.255
    or ((IP.s_bytes[1]=172) and (IP.s_bytes[2]>=16) and 
(IP.s_bytes[2]<=31))

    // 192.168.0.0 – 192.168.255.255
    or ((IP.s_bytes[1]=192) and (IP.s_bytes[2]=168));
end;

I didn't find a good unit in fcl-net to put it in.

IMO it would be better suited in sockets.pp in rtl-extra where in_addr 
is declared.


Feel free to add it there or in fcl-net, if you know a good place.

Thank you!
Ondrej

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

[fpc-devel] Function to find if IP address is local

[Ondrej Pokorny via fpc-devel]

Hello!

Is there a routine in RTL to find out if a given IPv4 address is from 
the private address range (10.* etc)?


Thanks
Ondrej

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] threads vs widestringmanager / crash

[Ondrej Pokorny via fpc-devel]

On 20.12.2022 21:12, Sven Barth via fpc-devel wrote:
Please stop focusing on the WideString manager. That is simply a 
symptom. The real cause is that a thread is created where there 
shouldn't be one and under normal circumstance there indeed will be none:
- if the initialization is executed as part of an application then 
there won't be any other thread, because the system initialization 
doesn't create one
- if the initialization is executed as part of a library then there 
won't be any other thread, because the initialization is run during 
the DLL_PROCESS_ATTACH call of the entry point which Windows executes 
with the global loader lock held, thus no thread can enter the entry 
point (which is also why there are some things you shouldn't do in a 
unit initialization)


The only cause for a thread to exist at that point is if another 
process calls CreateRemoteThread(Ex)() to create a thread in the 
process that is just initializing (Note: DebugBreakProcess() 
internally uses CreateRemoteThread(Ex)()). This means that the thread 
can interfere at *any* point of the initialization simply depending on 
the behavior of the Windows scheduler (as you hinted at in your other 
mail from today).


So the only logical solution is to stop the offending thread from 
executing or not to have it call InitThread() while the initialization 
section of the System unit is still running. The former might have 
unintended consequences (e.g. not being able to debug the unit 
initialization) while the later might work in most cases as long as 
that remote thread doesn't need to execute code that relies on 
InitThread().


Sven, thank you very much for the insights!

I would much appreciate a solution without my hacks.

Ondrej

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] threads vs widestringmanager / crash

[Ondrej Pokorny via fpc-devel]

On 19.12.2022 09:51, Michael Van Canneyt via fpc-devel wrote:
If you can prepare a simple sample program that shows the problem, I 
can try
to look into it. Maybe a fresh pair of eyes sees more than one pair of 
eyes?
You don't need a special sample program. Every console application on 
Windows will do. Just run it with debugging in Lazarus.


My observation is that the problem is worse on slow machines (virtual 
machines, shared VPS). I actually almost never observed it on my local 
PC, but on the VPS it made debugging nearly impossible.


I'll disable the "Assigned(WideStringManager.GetStandardCodePageProc)" 
hack on the VPS and report back.


Ondrej

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] threads vs widestringmanager / crash

[Ondrej Pokorny via fpc-devel]

On 19.12.2022 07:42, Sven Barth via fpc-devel wrote:
The RTL assumes that it does not have to deal with threads during the 
initialization of the System unit. So even if the symptom of the crash 
is fixed in that location it isn't said that something else might 
crash or that in the future code might be added that crashes...


Is it possible to acknowledge that the current RTLs assumption is wrong 
and it has to deal with threads during the System initialization? 
(Because obviously the assumption is wrong, otherwise there was no crash.)


That means not to fix the symptom with checking 
"Assigned(WideStringManager.GetStandardCodePageProc)" but acknowledge 
that the OS can start a thread before System initialization is done and 
create a mechanism to postpone thread initialization until System is 
fully initialized.


Something like:

    procedure InitThread(stklen:SizeUInt);
  begin
// new code begin
    while not SystemInitialized do
  Sleep(10);
// new code end
{$ifndef FPUNONE}
    SysResetFPU;
{$endif}

Ondrej

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] threads vs widestringmanager / crash

[Ondrej Pokorny via fpc-devel]

I have the same problem. It is very annoying because programs crash 
quite often when run in a debugger (on Windows).


I solved it with the attached patch - I added these lines to procedure 
OpenStdIO(var f:text;mode:longint;hdl:thandle);


  {$if defined(FPC_HAS_CPSTRING) and defined(FPC_HAS_FEATURE_WIDESTRINGS)}
    while not Assigned(WideStringManager.GetStandardCodePageProc) do
  ;
  {$endif}

The problem is that
procedure InitThread(stklen:SizeUInt);
is called before the WideStringManager properties are set in system 
initialization when debugger is present.


How to solve it properly? Can the InitThread() procedure somehow wait 
until the system initialization is done? E.g. with a global variable set 
with InterLockedIncrement()? Would such a patch be accepted in FPC?


I'd like to solve the problem on FPC level to avoid my own patches.

Thanks
Ondrej


Am 07.07.2018 um 14:40 schrieb Martin:

There actually isnt a thread started.

But during initialization "TThread.Queue" is called (because the code 
in question can be called in threads aswell, and must ensure work is 
done in the main thread)


"TThread.Queue" creates a new external thread. This fails during unit 
initialization for the reason below.


This leads me to the following questionQ

1) out of interest.
If "TThread.Queue" is called in the main thread, then it does not 
queue anyway, but rather run immediately.
In this case, why not avoid creating a thread, and check 
"CurrentThreadVar = nil" ?


2) how to solve my problem. (including other OS, for which I do not 
know if the issue exists)
- Is there a way to test, if current code is in the main thread? (and 
doing so, without creating a TThread instance)?


- If not, then for windows (and fpc 3.0.4) the following *should* do 
(but is really rather a hack)
  {$IFDEF MsWindows}  if WideStringManager.GetStandardCodePageProc = 
nil then


Any better ideas?



On 07/07/2018 14:00, Martin wrote:

fpc 3.0.4
Win 10 / 32bit

Lazarus trunk, but that may not be relevant.
This is the first time I am seeing this, and I am therefore not sure 
I will be able to reproduce, since it seems a race condition.


This appears to happen during unit initialization.

Something must have started a thread, before widestring manager got 
initialized.
In other word, some unit (used in such order that it's initialization 
runs before unit system initialization) has started a thread.


This thread crashes, if it is fast enough, because in the threads 
init it will access widestringmanager.


Is this a bug in fpc?
Or is this wrong of the calling code (eg starting threads during unit 
initialization).


In the latter case, is it documented?

The trace from the thread:
#0 ?? at :0
#1 OPENSTDIO({}, 55217, 168) at ..\inc\text.inc:2670
#2 SYSINITSTDIO at ..\win\syswin.inc:520
#3 INITTHREAD(16777216) at ..\inc\thread.inc:59
#4 EXEC_TLS_CALLBACK(0x40, 2, 0x0) at ..\win\systlsdir.inc:90
#5 ntdll!WinSqmSetDWORD64 at :0
#6 ?? at :0
#7 ntdll!RtlDeactivateActivationContextUnsafeFast at :0
#8 ntdll!RtlCaptureStackContext at :0
#9 ntdll!LdrShutdownThread at :0
#10 ntdll!LdrInitializeThunk at :0
#11 ntdll!LdrInitializeThunk at :0
#12 ntdll!LdrInitializeThunk at :0
#13 ?? at :0

The main thread sit right before widestring initialization.
#0 INITUNICODESTRINGMANAGER at ..\inc\ustrings.inc:2311
#1 SYSTEM_$$_init$ at :674
#2 fpc_initializeunits at ..\inc\system.inc:886
#3 main at lazarus.pp:87

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel


___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-develdiff --git 
"a/C:\\Users\\ONDREJ~1\\AppData\\Local\\Temp\\TortoiseGit\\text-1b4616c.000.inc"
 "b/P:\\software\\FPC\\trunk_src\\rtl\\inc\\text.inc"
index c2ef94fbaa..ddee0867f7 100644
--- 
"a/C:\\Users\\ONDREJ~1\\AppData\\Local\\Temp\\TortoiseGit\\text-1b4616c.000.inc"
+++ "b/P:\\software\\FPC\\trunk_src\\rtl\\inc\\text.inc"
@@ -2753,6 +2753,10 @@ procedure OpenStdIO(var f:text;mode:longint;hdl:thandle);
   TextRec(f).Handle:=hdl;
   TextRec(f).Mode:=mode;
   TextRec(f).Closefunc:=@FileCloseFunc;
+  {$if defined(FPC_HAS_CPSTRING) and defined(FPC_HAS_FEATURE_WIDESTRINGS)}
+while not Assigned(WideStringManager.GetStandardCodePageProc) do
+  ;
+  {$endif}
   case mode of
 fmInput :
   begin
___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] Sorting tests

[Ondrej Pokorny via fpc-devel]

Am 29.11.2022 um 14:25 schrieb Sven Barth via fpc-devel:
For such a comparison function the issue is indeed in the comparison 
function, but Nikolay also mentioned "ais the case for equal elements.


I assume this was some kind of typo or reasoning error because 
"acompare function is correct. And Nikolay was talking about incorrect 
comparison functions.


Ondrej
___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] Sorting tests

[Ondrej Pokorny via fpc-devel]

Am 29.11.2022 um 11:08 schrieb Sven Barth via fpc-devel:
J. Gareth Moreton via fpc-devel  
schrieb am Di., 29. Nov. 2022, 10:09:


Surely that's a bug in the comparison functions that should be
fixed and
not something that can be blamed on introsort.  If a comparison
function
is faulty, then pretty nuch any sorting algorithm can be
considered to
have unpredictable behaviour.


This *can* be blamed on IntroSort, because Stability (order of equal 
elements is kept) is an attribute of sorting algorithms and IntroSort 
is *not* considered stable while QuickSort *can* be stable depending 
on the implementation and ours *is*.


If for two elements [a, b] the comparison function returns
athen the problem is not in stability or any other feature of the sorting 
algorithm but in the comparison function indeed. Or am I missing something?


Ondrej
___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] Just need to double check the size of Integer

[Ondrej Pokorny via fpc-devel]

Am 24.11.2022 um 14:44 schrieb Tomas Hajny via fpc-devel:

On 2022-11-24 14:25, Ondrej Pokorny via fpc-devel wrote:

Am 24.11.2022 um 14:15 schrieb Marco van de Voort via fpc-devel:

On 24-11-2022 14:13, J. Gareth Moreton via fpc-devel wrote:


I just need to double-check something.  Under CPU16, SizeInt and 
SizeUInt are declared as follows:


  SizeInt = Integer;
  SizeUInt = Word;

Is Integer 16-bit in this case?


Yes.


Are you sure? systemh.inc is compiled with {$mode objfpc} (see line
22), so Integer within the system unit is 32-bit.

So to me it looks like a bug.


No, it isn't a bug - systemh.inc is indeed compiled in $MODE OBJFPC 
nowadays, but it defines Integer = SmallInt on line 110. Depending on 
compilation mode used for your sources (containing references to 
Integer), the definition in unit objpas (included automatically for 
modes objfpc and delphi) may override the definition in systemh.inc.


All clear now. Thanks!

Ondrej

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] Just need to double check the size of Integer

[Ondrej Pokorny via fpc-devel]

Am 24.11.2022 um 14:15 schrieb Marco van de Voort via fpc-devel:


On 24-11-2022 14:13, J. Gareth Moreton via fpc-devel wrote:


I just need to double-check something.  Under CPU16, SizeInt and 
SizeUInt are declared as follows:


  SizeInt = Integer;
  SizeUInt = Word;

Is Integer 16-bit in this case? 


Yes.

I always thought Integer was defined as a signed 32-bit integer under 
FreePascal. 


No. Only in objfpc/delphi modes, and system is in $mode FPC. Unit 
objpas overrides this definition and is loaded in objfpc/delphi mode.


OK, I found it. Actually within the system unit Integer is always 16-bit 
because it is re-declared on line 114:

Integer  = SmallInt;

So yes, Integer in the system unit is 16-bit but because of the 
re-declaration and not the mode.


Ondrej

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] Just need to double check the size of Integer

[Ondrej Pokorny via fpc-devel]

Am 24.11.2022 um 14:15 schrieb Marco van de Voort via fpc-devel:

On 24-11-2022 14:13, J. Gareth Moreton via fpc-devel wrote:


I just need to double-check something.  Under CPU16, SizeInt and 
SizeUInt are declared as follows:


  SizeInt = Integer;
  SizeUInt = Word;

Is Integer 16-bit in this case? 


Yes.


Are you sure? systemh.inc is compiled with {$mode objfpc} (see line 22), 
so Integer within the system unit is 32-bit.


So to me it looks like a bug.

Documentation:
https://www.freepascal.org/docs-html/ref/refsu4.html
https://www.freepascal.org/docs-html/rtl/system/sizeint.html

Ondrej

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] Compile error with actual FPC main and Lazarus main

[Ondrej Pokorny via fpc-devel]

Am 28.09.2022 um 18:53 schrieb Gabor Boros via fpc-devel:

2022.09.28. 17:43 keltezéssel, Ondrej Pokorny via fpc-devel írta:

Try a clean build.

Ondrej


Clean build is ok. The problem appears when use "Build Lazarus with 
Profile: Normal IDE".


That is quite normal. If you see such errors, check the "clean all" 
checkbox in the Build Lazarus dialog.


Ondrej


___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] Compile error with actual FPC main and Lazarus main

[Ondrej Pokorny via fpc-devel]

Am 28.09.2022 um 12:53 schrieb Gabor Boros via fpc-devel:

Hi All,

With actual FPC (123f196d) and Lazarus (11bb7ce4) got the below error. 
Any idea?


sourcefilemanager.pas(311,1) Error: Compilation raised exception 
internally


Try a clean build.

Ondrej

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

[fpc-devel] Fwd: [Lazarusdev] Moving issue 39853 to FPC project

[Ondrej Pokorny via fpc-devel]

Hello it looks like only people with FPC developer status can move 
issues to the FPC project. Therefore I forward the request to fpc-devel.


Ondrej


 Weitergeleitete Nachricht 
Betreff:[Lazarusdev] Moving issue 39853 to FPC project
Datum:  Thu, 18 Aug 2022 03:51:33 +0300
Kopie (CC): Maxim Ganetsky 



Hello.

Could someone move this issue to FPC project please:
https://gitlab.com/freepascal.org/lazarus/lazarus/-/issues/39853

--
Best regards,
 Maxim Ganetskymailto:gan...@narod.ru
___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] Gaps in a non-contiguous enum

[Ondrej Pokorny via fpc-devel]

Am 16.08.2022 um 13:19 schrieb Michael Van Canneyt via fpc-devel:

On Tue, 16 Aug 2022, Ondrej Pokorny via fpc-devel wrote:

--- code end

Run it in Delphi and FPC and compare the outputs:

Delphi output:
PropCount: 0
Property info is null

FPC output:
PropCount: 1
Property name: Fruit
Property kind: tkEnumeration

As you can see FPC creates a valid PropInfo for the property with 
valid TypeInfo (PI^.PropType) of the kind tkEnumeration! This is 
definitely wrong.


So the solution is either:

1.) don't create PropInfo for the Fruit property like Delphi.
- or -
2.) PropType must not be thEnumeration. It should be either tkInteger 
or a new custom value like tkCEnumeration or whatever. Because 
thEnumeration means it has the enumeration TypeInfo with GetEnumName 
and GetEnumValue.


tkInteger or no propinfo at all seems the only correct solution to me, 
with

slight preference for no propinfo at all.
Sorry I forgot - a third solution that you and Martin suggested before 
is also OK:
3.) compile error when the Fruit property is published (it is allowed 
only in the private..public sections).


So, I suggest:

*) Delphi mode:  ignore the PropInfo (Delphi compatibility)
*) ObjFpc mode: compiler error (because you prefer no propinfo at all 
and ObjFpc is usually more strict than Delphi).


I've done some PropInfo work before, I assume I could prepare a patch 
for it. Can I start with this work?


Ondrej

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] Gaps in a non-contiguous enum

[Ondrej Pokorny via fpc-devel]

Am 16.08.2022 um 09:37 schrieb Michael Van Canneyt via fpc-devel:

On Tue, 16 Aug 2022, Juha Manninen via fpc-devel wrote:

On Thu, Aug 11, 2022 at 9:37 PM Martin Frb via fpc-devel <
fpc-devel@lists.freepascal.org> wrote:


fpc code is still "unexpected".
It takes into account the MinValue

type TMyEnum = (a=5, b=7);

GetEnumName(pt, 5) // a
GetEnumName(pt, 6) // b // should be 7

if it is just the nth name, then it should start at 0 (or if needs 
be at

1).



And function GetEnumValue() is buggy.


It is not buggy. It does not support enumerateds with gaps, because 
RTTI does not support enumerateds with gaps.

It never has.


Well it is buggy, but a little bit differently: FPC indeed creates 
PropInfo for enumerated types with gaps!


Allow me to slightly edit your demo code:

program RTTITest;

{$ifdef DCC}
// Delphi
{$APPTYPE CONSOLE}
{$else}
// FPC
{$mode objfpc}{$h+}
{$endif}

uses
  SysUtils, Classes, TypInfo;

type
  TFruitMore = (fmApple=1, fmOrange=3, fmBanana=5, fmGrapes, fmPear);

  TMyClass = class(TPersistent)
  private
    FFruit: TFruitMore;
  Published
    property Fruit : TFruitMore Read FFruit Write FFruit;
  end;

var
  PI : PPropInfo;
  aClass : TMyClass;
  TypeData : PTypeData;
begin
  aClass:=TMyClass.Create;
  TypeData:=GetTypeData(aClass.ClassInfo);
  Writeln('PropCount: ', TypeData^.PropCount); // Delphi writes 0, FPC 
writes 1


  PI:=GetPropInfo(aClass,'Fruit',tkAny);
  if PI=nil then
    Writeln('Property info is null')
  else
  begin
    Writeln('Property name: ', PI^.Name);
    Writeln('Property kind: ', GetEnumName(TypeInfo(TTypeKind), 
Ord(PI^.PropType^.Kind)));

  end;

  Readln;
end.

--- code end

Run it in Delphi and FPC and compare the outputs:

Delphi output:
PropCount: 0
Property info is null

FPC output:
PropCount: 1
Property name: Fruit
Property kind: tkEnumeration

As you can see FPC creates a valid PropInfo for the property with valid 
TypeInfo (PI^.PropType) of the kind tkEnumeration! This is definitely wrong.


So the solution is either:

1.) don't create PropInfo for the Fruit property like Delphi.
- or -
2.) PropType must not be thEnumeration. It should be either tkInteger or 
a new custom value like tkCEnumeration or whatever. Because 
thEnumeration means it has the enumeration TypeInfo with GetEnumName and 
GetEnumValue.


Ondrej

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] tkLString vs tkAString

[Ondrej Pokorny via fpc-devel]

On 30.11.2021 13:43, Marc Weustink via fpc-devel wrote:

On 30-11-2021 13:37, Michael Van Canneyt via fpc-devel wrote:

On Tue, 30 Nov 2021, Marc Weustink via fpc-devel wrote:

Hi,

I ran into an issue with a piece of ported Delphi (6) code and RTTI. 
Part of the Delphi code was based on strings with a RTTI type 
tkLString where in FPC the type appeared to be tkAString. Fixing 
this was easy, but it made me wonder what the difference is between 
them.

From the documentation:
 Source position: rttih.inc line 22
  type TTypeKind = (
  ...
  tkLString,  Longstring property.
  tkAString,  Ansistring property.


So there is apparently a LongString type and an AnsiString type. 
However if you search (google) the documentation on LongString, you 
only find references to AnsiString.


My question: What is the difference (if any) ?


This dates from long ago...

Longstring was an idea for a shortstring/ansistring kind of string - 
unlimited length but no reference counting.


But it was never fully implemented.


Ah, that explains. So technically the delphi tkLString is the fpc 
tkAString and the fpc tkLString does not exist.


Yes: 
https://docwiki.embarcadero.com/Libraries/Sydney/en/System.TypInfo.TTypeKinds


The RTTI in FPC differs a little bit from the Delphi's and therefore 
IFDEFs are needed in shared code. E.g. tkBool and tkUChar from FPC don't 
exist in Delphi either.


I assume MvC knows more details - that is just what I stumped upon.

Ondrej

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] OpenSSL v1.1.x DLL loading / Bug 37137

[Ondrej Pokorny via fpc-devel]

On 28.09.2020 16:05, Michael Van Canneyt wrote:

On Mon, 28 Sep 2020, Ondrej Pokorny via fpc-devel wrote:


On 28.09.2020 15:34, Michael Van Canneyt via fpc-devel wrote:

I made some time for this, it's checked in with revision 46987.

Note that this revision also enables support for LibreSSL (or 
whatever it is

called) versions on linux/mac systems.

Thank you for your patch, and I'm sorry that it took me so long to 
get to it...


Hello,

openssl doesn't build anymore (make fpc trunk):

Compiling openssl\BuildUnit_openssl.pp

Compiling FPC\trunk_src\packages\openssl\src\openssl.pas
openssl.pas(5629,66) Error: Wrong number of parameters specified for 
call to "MakeLibName"
openssl.pas(5570,11) Error: Found declaration: MakeLibName(const 
AnsiString;const AnsiString):AnsiString(0);

openssl.pas(5710) Fatal: There were 2 errors compiling module, stopping
Fatal: Compilation aborted


Should be fixed in rev. 46991.


Thank you!

Ondrej

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] OpenSSL v1.1.x DLL loading / Bug 37137

[Ondrej Pokorny via fpc-devel]

On 28.09.2020 15:34, Michael Van Canneyt via fpc-devel wrote:

I made some time for this, it's checked in with revision 46987.

Note that this revision also enables support for LibreSSL (or whatever 
it is

called) versions on linux/mac systems.

Thank you for your patch, and I'm sorry that it took me so long to get 
to it...


Hello,

openssl doesn't build anymore (make fpc trunk):

Compiling openssl\BuildUnit_openssl.pp

Compiling FPC\trunk_src\packages\openssl\src\openssl.pas
openssl.pas(5629,66) Error: Wrong number of parameters specified for 
call to "MakeLibName"
openssl.pas(5570,11) Error: Found declaration: MakeLibName(const 
AnsiString;const AnsiString):AnsiString(0);

openssl.pas(5710) Fatal: There were 2 errors compiling module, stopping
Fatal: Compilation aborted

Ondrej
___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] Another thread about the fact that official FPC releases are *unnecessarily* non-representative of the platforms it actually runs on

[Ondrej Pokorny via fpc-devel]

On 27.09.2020 23:45, Travis Siegel via fpc-devel wrote:

On 9/27/2020 3:50 PM, Florian Klämpfl via fpc-devel wrote:

- there are little libraries being really IEEE compliant for float128

- if they are IEEE compliant, their license does not allow to use the
code in the FPC rtl.



I don't understand this statement.

How does having all the required functions for a certain kind of 
compliance restrict anything?  I was under the impression that if 
someone wrote those functions from scratch that there was no licensing 
issue.  Where's the conflict in that case?


I've read most of the gpl licenses multiple times, and I've never seen 
anything targeting compliance of any kind, What does this statement 
stem from, and how does it apply to FPC code specifically?


This would seem to indicate that no gpl license code could ever obtain 
compliance with certain standards, and I've seen more than once where 
said code does just that.  I don't get it.


Someone enlighten me please.


It just means that all IEEE compliant libraries that Florian knows are 
released under a license that does not allow to include them within the FPC.


Ondrej

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] TThread.RemoveQueuedEvents

[Ondrej Pokorny via fpc-devel]

On 25.09.2020 03:30, Martin via fpc-devel wrote:

Wath is in the following indented?

The doc says 
https://www.freepascal.org/docs-html/rtl/classes/tthread.removequeuedevents.html
all calls to the given method  are removed. 


However there may be more than one way to read this.

aMethod: TThreadMethod 


is "procedure of object"

The method "aMethod" (which is what is given) therefore consists of 
proc-address and instance-address.


However the code actually removes all methods at proc-address, 
regardless of the instance.


Does the "given method" mean:
1) "aMethod" as passed in as parameter, consisting of proc and instance
2) "method of the class" extracted from "aMethod"


It's a Delphi-compatibility method: 
http://docwiki.embarcadero.com/Libraries/Sydney/en/System.Classes.TThread.RemoveQueuedEvents


The Delphi docs are not much more help here but FPC should have the same 
behavior. Please check the Delphi behavior and clarify the docs.


Thanks, Ondrej

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] about #0035991, fpc 3.2.1 still crash lazarus 2.0.10, 2.0.11

[Ondrej Pokorny via fpc-devel]

Hello Bald,

it is not possible to support the latest FPC trunk in already released 
Lazarus versions (2.0.10).


If you want some bug fixes to be merged into the stable branch 
(currently 2.0.11), you can list the revisions here:

https://wiki.freepascal.org/Lazarus_2.0_fixes_branch

I did it for you now for the two revisions of issue 35991.

Ondrej


On 01.09.2020 03:21, baldzhang via fpc-devel wrote:

Hi, Ondrej

I saw you mark #0035991 resolved, but I still has crash problem on 
Svn2RevisionInc.

The testing result is:
 3.2.0
   2.0.10
   2.0.11
   2.1 : all good

 3.2.1
   2.0.10  : exception
   2.0.11  : exception
   2.1 : good

This was tested with a small program copy from Svn2RevisionInc, and compiled 
with different version's lazutils source codes.
(source code attached)

The exceptions are raised from same location, a dump from svn2revisioninc looks 
like:

An unhandled exception occurred at $0001A13A:
EAccessViolation: Access violation
   $0001A13A fpc_ansistr_decr_ref, line 146 of ../inc/astrings.inc
   $000100012921 fpc_finalize, line 234 of ../inc/rtti.inc
   $000100012657 RECORDRTTI, line 134 of ../inc/rtti.inc
   $000100010C91 CLEANUPINSTANCE, line 755 of ../inc/objpas.inc
   $000100010489 FREEINSTANCE, line 446 of ../inc/objpas.inc
   $00010004529A DESTROY, line 277 of fcl-process/src/processbody.inc
   $00011962 fin$0012, line 118 of svn2revisioninc.pas
   $000119ED SVNINPATH, line 109 of svn2revisioninc.pas
   $00013C91 PARAMSVALID, line 572 of svn2revisioninc.pas
   $000146DD GITREVISIONFROMGITCOMMIT, line 769 of svn2revisioninc.pas
   $000147C7 main, line 788 of svn2revisioninc.pas
   $000147E6 MAIN_WRAPPER, line 126 of system.pp
   $00010001AE27 EXE_ENTRY, line 240 of system.pp
   $0001192D _FPC_MAINCRTSTARTUP, line 106 of sysinit.pp
   $7FFB4D7B7BD4
   $7FFB4E96CE51


I write some note in bug tracker, you may not noticed yet.

So my question is:
Is there any plan to fix it with this bundle(fpc 3.2.x/lazarus 2.0.x)? Or 
lazarus 2.0.x will stick on fpc 3.2.0?

Because I need fpc 3.2.1 with new ssl features but did not  want to maintain 
extra patches when building fpc/lazarus.

Thanks.
B. Zhang

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] ftfont and FreeType

[Ondrej Pokorny]

I am not 100% sure, but some freetype.dll versions can depend on 
zlib1.dll. I.e. you need to copy zlib1.dll to the exe directory as well.


Actually I just found an info about it: 
https://wiki.freepascal.org/FPReport_FAQ (see Lazarus does not start 
anymore in Windows).


Ondrej


On 17.07.2020 11:37, Werner Pamler wrote:
I am having issues in the nogui demo of TAChart with FPC 3.2.0. It 
uses the FCL unit ftfont and crashes at runtime with


    Project noguidemo.exe raised exception class 'EInOutError' with 
message:
    Can not load Freetype library "freetype.dll". Check your 
installation.


This does not happen when I compile exactly the same program with FPC 
3.0.4


Since I see in the ftfont code the directive {$DEFINE DYNAMIC} I 
downloaded the freetype.dll from the 
https://github.com/ubawurinna/freetype-windows-binaries (recommended 
by freetype.org) into the exe directory, but it still does not run 
(same error).


Any idea what I can do to make the program work again? I am on Windows 
10.


___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

[fpc-devel] AS-IS for enums

[Ondrej Pokorny]

Hello Florian,

as we spoke I checked the patch for IS operators for enumerations. I 
also added int64/dword value compatibility that was missing in the last 
patches.


There is still some bug/issue left. The objfpc mode shows warnings like 
this:

test_as_is_1.lpr(51,8) Warning: Expectloc is not set in firstpass: calln

and the delphi mode crashes with an internal error 200306031.

I don't remember that this bug happened the last time I was working on 
this. Could you please take a look at it?


I also added the check for enums with holes in objfpc mode that disables 
the IS operators there. It is marked with a ToDo comment. If you 
decide to keep it there we need a new error message. If you decide to 
delete it we don't need a new error message. I would personally delete 
the extra check and enable AS also for enums with holes (checking 
only boundaries) but I don't care that much.


I attach the latest patch and a test program.

Thanks
Ondrej

program test_as_is_1;

{$define usedelphi}

{$ifdef usedelphi}
  {$mode delphi}
{$else}
  {$mode objfpc}
{$endif}

uses SysUtils;

type
  TMyEnum = (zero, one, two);
  TMyEnum2 = (zero2, one2, two2);
{$ifdef usedelphi}
  TMyHoles = (zeroh, twoh = 2);
{$endif}

var
  E: TMyEnum;
  E2: TMyEnum2;
{$ifdef usedelphi}
  H: TMyHoles;
{$endif}
  I: Integer;
  
begin
  // check a valid integer value agains an enum type
  I := 1;
  if I is TMyEnum then
E := I as TMyEnum
  else
E := zero;
  if E<>one then
Halt(1);

  // check an invalid integer value agains an enum type
  I := 5;
  if I is TMyEnum then
Halt(2);
  try
E := I as TMyEnum;
Halt(3);
  except
E := one;
  end;

  // check an enum value against an enum type
  E := one;
  if E is TMyEnum2 then
E2 := E as TMyEnum2
  else
E2 := zero2;
  if E2<>one2 then
Halt(4);

  // check a const enum value agaist an enum type
  if one is TMyEnum2 then
E2 := one as TMyEnum2
  else
E2 := zero2;
  if E2<>one2 then
Halt(5);

  // check an enum variable against the same enum type - is always true due to 
optimizations !!!
  E := TMyEnum(-1);
  if E is TMyEnum then
  begin
if Ord(E) is TMyEnum then
  Halt(7);
  end else
Halt(8);

{$ifdef usedelphi}
  // check an ord value agains an enum type with holes
  I := 1;
  if I is TMyHoles then // 1 is a valid value of TMyHoles in delphi mode
H := I as TMyHoles
  else
H := zeroh;
  if H<>TMyHoles(1) then
Halt(9);
{$endif}

  Writeln('all good');
  ReadLn;
end.

Index: compiler/ncnv.pas
===
--- compiler/ncnv.pas   (revision 44280)
+++ compiler/ncnv.pas   (working copy)
@@ -4376,6 +4376,20 @@
   end;
   end;
   end
+else if (right.resultdef.typ=enumdef) then
+  begin
+{ left must be an ordinal }
+if not is_ordinal(left.resultdef) then
+  CGMessage1(type_e_ordinal_expr_expected,left.resultdef.typename);
+case nodetype of
+  isn:
+resultdef:=pasbool1type;
+  asn:
+resultdef:=right.resultdef;
+  else
+InternalError(2019070201);
+end;
+  end
 else
   
CGMessage1(type_e_class_or_interface_type_expected,right.resultdef.typename);
   end;
@@ -4404,6 +4418,9 @@
 procname: string;
 statement : tstatementnode;
 tempnode : ttempcreatenode;
+v,res: Tconstexprint;
+leftconv: tnode;
+resulttype: tordtype;
   begin
 result:=nil;
 { Passing a class type to an "is" expression cannot result in a class
@@ -4449,6 +4466,61 @@
 ccallparanode.create(left,ccallparanode.create(right,nil)),
 resultdef);
   end
+else if (right.resultdef.typ=enumdef) then
+  begin
+if tenumdef(right.resultdef).has_jumps and not (m_delphi in 
current_settings.modeswitches) then // ToDo: maybe delete this check to enable 
enums with holes in objfpc
+  Message(type_e_succ_and_pred_enums_with_assign_not_possible); // 
ToDo: new error message
+if left.nodetype=ordconstn then
+  begin // 1 is TEnum: left is ordinal constant - do the check in 
compile-time
+v:=Tordconstnode(left).value;
+res.signed:=true;
+res.overflow:=false;
+res.svalue:=Ord((v>=tenumdef(right.resultdef).min) and 
(v<=tenumdef(right.resultdef).max));
+result:=cordconstnode.create(res, resultdef, false);
+  end
+else
+if (left.resultdef.typ=enumdef) and 
(tenumdef(right.resultdef)=tenumdef(left.resultdef)) then
+  begin // the value is the same enumeration type - ignore the 
check
+v:=Tordconstnode(left).value;
+res.signed:=true;
+res.overflow:=false;
+

Re: [fpc-devel] TEncoding.Default and default encoding for TStrings.LoadFrom*()

[Ondrej Pokorny]

On 03.01.2020 11:09, Michael Van Canneyt wrote:
I also think it is very hypothetical, and not a problem unless you 
want to use

the same stream in Delphi and FPC.

Well, you have my blessing for the soPreserveBOM :)


Added in r43848. I'll check how the FPC documentation works and try to 
add it there.


Ondrej

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] A more fundamental problem Re: i36507

[Ondrej Pokorny]

On 03.01.2020 11:00, J. Gareth Moreton wrote:
This was not a new property or a recent change to the Lazarus sources 
though.


It was a recent change in FPC RTL sources: r43841.


I was using an old revision of Lazarus to test optimisations on, and 
had been doing fine with it until I discovered last night that it 
would no longer build.  Updating to the latest repository didn't 
change the file in question. Something in the compiler was changed 
recently that caused it to break when it was able to handle the clash 
before.


There was no clash before. The Options property was introduced yesterday 
in r43841.


FPC trunk is supported only in Lazarus trunk (and Lazarus stable branch 
if not forgotten :) ).


Ondrej

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] A more fundamental problem Re: i36507

[Ondrej Pokorny]

On 03.01.2020 10:45, J. Gareth Moreton wrote:
Given the "illegal qualifier" errors that appeared, I'm guessing the 
issue is that "Options" is also a property of the THistoryList class 
as well as a local variable.  Technically speaking though, this is a 
regression unless it's to be documented (and hence break backward 
compatibility) that if you have a property and a local variable of the 
same name, the property takes precedence.


Nonsense.

That would mean that adding any new property in any object is a breaking 
change because the same name can be used in any possible with-statement 
at any place in any program or any local variable etc.


Short: every new property introduces a risk of clashes.

Ondrej

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] TEncoding.Default and default encoding for TStrings.LoadFrom*()

[Ondrej Pokorny]

On 03.01.2020 10:14, Michael Van Canneyt wrote:

On Fri, 3 Jan 2020, Ondrej Pokorny wrote:


On 03.01.2020 00:35, Werner Pamler wrote:

Am 02.01.2020 um 20:10 schrieb Ondrej Pokorny:
TStrings.SaveTo*() writes BOM by default. Formerly the BOM was not 
written.


There is also the problem that currently it is not possible, without 
further action, to retain the BOM state of a file loaded into a 
stringlist, modified and written back because the presence of a BOM 
is forgotten after reading - see the other discussion
(https://lists.freepascal.org/pipermail/fpc-devel/2020-January/042363.html). 

Wouldn't it make sense to introduce a )Read)BOM property (a boolean 
parameter or an element of the new Options) which gets its value 
when the file is loaded or the strings are assigned? Then the user 
can set the StringList.WriteBOM equal to the StringList.BOM when he 
wants to keep the BOM for writing back.


Yes, that is perfectly reasonable. I'd prefer a new element in 
Options but there is the risk that Delphi adds a new option in the 
future and then we'll have a problem. So maybe a "PreserveBOM" or 
"SetWriteBOMOnLoad" property will be better. When set to true, 
WriteBOM will be set in LoadFrom*() according to BOM presence of the 
loaded file.


I don't see why a new option is a problem ? They are not streamed anyway.

So I would do the opposite, add an option. soPreserveBOM.


If you are fine with it, me as well.

Yes, the problem is if somebody streams the property or uses 
Ord(soPreserveBOM) for something etc. I admit that it is a very 
hypothetical issue.


Ondrej

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] TEncoding.Default and default encoding for TStrings.LoadFrom*()

[Ondrej Pokorny]

On 03.01.2020 00:35, Werner Pamler wrote:

Am 02.01.2020 um 20:10 schrieb Ondrej Pokorny:
TStrings.SaveTo*() writes BOM by default. Formerly the BOM was not 
written.


There is also the problem that currently it is not possible, without 
further action, to retain the BOM state of a file loaded into a 
stringlist, modified and written back because the presence of a BOM is 
forgotten after reading - see the other discussion 
(https://lists.freepascal.org/pipermail/fpc-devel/2020-January/042363.html). 
Wouldn't it make sense to introduce a )Read)BOM property (a boolean 
parameter or an element of the new Options) which gets its value when 
the file is loaded or the strings are assigned? Then the user can set 
the StringList.WriteBOM equal to the StringList.BOM when he wants to 
keep the BOM for writing back.


Yes, that is perfectly reasonable. I'd prefer a new element in Options 
but there is the risk that Delphi adds a new option in the future and 
then we'll have a problem. So maybe a "PreserveBOM" or 
"SetWriteBOMOnLoad" property will be better. When set to true, WriteBOM 
will be set in LoadFrom*() according to BOM presence of the loaded file.


Ondrej

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] Make all fails after revision 43841 (Ondrej)

[Ondrej Pokorny]

It should be fixed with r43846. You may need to delete fpmake.exe and 
fpmake.o to make sure they are recompiled.


Ondrej

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] Make all fails after revision 43841 (Ondrej)

[Ondrej Pokorny]

On 02.01.2020 20:34, Bart via fpc-devel wrote:

After revision 43841 make all in the root fails


Thanks - I fixed this one. But there seems to be another problem. I am 
on it now.


Ondrej

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] TEncoding.Default and default encoding for TStrings.LoadFrom*()

[Ondrej Pokorny]

On 27.12.2019 12:01, Ondrej Pokorny wrote:

On 27.12.2019 10:40, Michael Van Canneyt wrote:

Yes, indeed. Therefore I suggested
* TEncoding.Default for the DefaultSystemCodePage variable
and
* TEncoding.ANSI for the system encoding.

Currently we have
* TEncoding.SystemEncoding for the DefaultSystemCodePage variable
and
* both TEncoding.ANSI and TEncoding.Default for the system encoding. 
(TEncoding.ANSI and TEncoding.Default are equal in FPC.)


In that case,  why not simply change:

 class function TEncoding.GetDefault: TEncoding;
 begin
   Result := GetSystemEncoding;
 end;

Nothing need be removed. I consider SystemEncoding a better name than 
Default,
and the latter should only be kept for Delphi compatibility. IMHO it 
would be

better to avoid Default, in fact I would change references to Default to
SystemEncoding for clarity. Default is completely non-descriptive.

If I understand your reasoning correct, that should solve the 
problems you

report ?


Yes, that perfectly solves the issues Lazarus developers and users 
face. I am OK with this solution as well. Thanks!


I applied the change

class function TEncoding.GetDefault: TEncoding;
 begin
   Result := GetSystemEncoding;
 end;

in r43842 before it gets forgotten. I removed the ANSI-hack from Lazarus 
as well - in r62474.


Please note that in Lazarus (where the system encoding is UTF-8), 
TStrings.SaveTo*() writes BOM by default. Formerly the BOM was not 
written. Bart reported the issue here: 
https://lists.freepascal.org/pipermail/fpc-devel/2020-January/042372.html


Ondrej

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] TStrings and BOM

[Ondrej Pokorny]

On 02.01.2020 14:58, Michael Van Canneyt wrote:

On Thu, 2 Jan 2020, Bart via fpc-devel wrote:


On Thu, Jan 2, 2020 at 12:12 PM Michael Van Canneyt
 wrote:


In Delphi it doesn't matter where the encoding comes from.
If writeBOM is true and there is a preamble for the encoding 
(whatever it

is), it is used.


I think I have to rephrase my question.
In fpc TStrings.SaveToStream(AStream) does NOT use DefaultEncoding if
Encoding is nil.
So this code:

//Source codepage is Windows 1512
SL := TStringList.Create;
SL.SkipLastLineBreak := True;
SL.WriteBOM := True;
SL.DefaultEncoding := TEncoding.UTF8;
S := 'ä'; // = #$0E
SL.Add(S);
SL.SaveToStream(St)

Will result in St having as content 1 single byte: 0E (hex)

Whereas
SL.SaveToStream(St, nil) will result in St having a content of 5
bytes: EF BB BF C3 A4

Is that Delphi compatible?
From the help text of the Embarcadero site I cannot figure that out.


That seems not compatible to me.


Should be fixed in r43839. Please report any issues.

Ondrej

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] TStrings and BOM

[Ondrej Pokorny]

On 02.01.2020 0:51, Ondrej Pokorny wrote:

On 01.01.2020 23:59, Werner Pamler wrote:

Am 01.01.2020 um 17:01 schrieb Ondrej Pokorny:
Isn't the TStrings.WriteBOM property good enough? Why to have yet 
another not-very-useful overload?


To be honest I think Bart's idea of making the BOM an optional 
parameter of the Save methods appears to me more efficient than 
defining a property just for this only purpose. Do we really have to 
imitate all the nonsense dictated by Delphi?


I see it's quite popular to humble every new Delphi feature after 
Delphi 7 as being nonsense. Not every time the criticism is valid. And 
never it is a good argument.


The WriteBOM property has a valid purpose - it is set in LoadFrom* 
according to the BOM presence in the loaded file. I.e. if you do:


MyStrings.LoadFromFile('abc'); // WriteBOM and Encoding properties are 
set here
MyStrings.SaveToFile('abc'); // they are used here to save the file as 
it was loaded


The abc file keeps BOM and encoding of the original one. You cannot 
achieve it with the AWriteBOM parameter Bart suggested.


I stand corrected. Neither FPC nor Lazarus do it. Obviously my memory 
doesn't serve me well and I didn't check it before sending the email. 
Only the encoding is set, not the WriteBOM property.


But still adding an overload just to omit one line for changing some 
property is nonsense. You can do so in a class helper if you want.


Ondrej

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] TStrings and BOM

[Ondrej Pokorny]

On 01.01.2020 23:59, Werner Pamler wrote:

Am 01.01.2020 um 17:01 schrieb Ondrej Pokorny:
Isn't the TStrings.WriteBOM property good enough? Why to have yet 
another not-very-useful overload?


To be honest I think Bart's idea of making the BOM an optional 
parameter of the Save methods appears to me more efficient than 
defining a property just for this only purpose. Do we really have to 
imitate all the nonsense dictated by Delphi?


I see it's quite popular to humble every new Delphi feature after Delphi 
7 as being nonsense. Not every time the criticism is valid. And never it 
is a good argument.


The WriteBOM property has a valid purpose - it is set in LoadFrom* 
according to the BOM presence in the loaded file. I.e. if you do:


MyStrings.LoadFromFile('abc'); // WriteBOM and Encoding properties are 
set here
MyStrings.SaveToFile('abc'); // they are used here to save the file as 
it was loaded


The abc file keeps BOM and encoding of the original one. You cannot 
achieve it with the AWriteBOM parameter Bart suggested.


If you want to simplify a call, use a class helper:

TStringsHelper = class helper for TStrings
...
procedure TStringsHelper.SaveToFileWithBOM(aFileName: string);
begin
  WriteBOM := True;
  SaveToFile(aFileName);
end;

Ondrej


___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] TStrings and BOM

[Ondrej Pokorny]

On 01.01.2020 13:00, Bart via fpc-devel wrote:

On Wed, Jan 1, 2020 at 12:25 PM Bart  wrote:


By all means, patches welcome.

Patch attached.
For completeness I could implement also SaveToStream(AStream,
AWriteBom) and SaveToFile(FileName, AWriteBom).


// code begin //
-Procedure TStrings.SaveToStream(Stream: TStream; AEncoding: TEncoding);
-
 Var B : TBytes;

 begin
   if AEncoding=nil then
 AEncoding:=FDefaultEncoding;
*-  if FWriteBOM then**
**+  if AWriteBOM then**
* begin
   B:=AEncoding.GetPreamble;
   if Length(B)>0 then
// code end //


You replace the WriteBOM property with the AWriteBOM parameter. What's 
the point of it? What do you plan to do with the WriteBOM property then?


Ondrej

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] TStrings and BOM

[Ondrej Pokorny]

On 31.12.2019 18:37, Michael Van Canneyt wrote:

On Tue, 31 Dec 2019, Bart via fpc-devel wrote:

Would introducing a overload of TStrings.SaveTo* with a 3rd parameter
to controle the writing of the BOM be acceptable?


By all means, patches welcome.


Isn't the TStrings.WriteBOM property good enough? Why to have yet 
another not-very-useful overload?


Ondrej

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] TEncoding.Default and default encoding for TStrings.LoadFrom*()

[Ondrej Pokorny]

On 27.12.2019 10:40, Michael Van Canneyt wrote:

Yes, indeed. Therefore I suggested
* TEncoding.Default for the DefaultSystemCodePage variable
and
* TEncoding.ANSI for the system encoding.

Currently we have
* TEncoding.SystemEncoding for the DefaultSystemCodePage variable
and
* both TEncoding.ANSI and TEncoding.Default for the system encoding. 
(TEncoding.ANSI and TEncoding.Default are equal in FPC.)


In that case,  why not simply change:

 class function TEncoding.GetDefault: TEncoding;
 begin
   Result := GetSystemEncoding;
 end;

Nothing need be removed. I consider SystemEncoding a better name than 
Default,
and the latter should only be kept for Delphi compatibility. IMHO it 
would be

better to avoid Default, in fact I would change references to Default to
SystemEncoding for clarity. Default is completely non-descriptive.

If I understand your reasoning correct, that should solve the problems 
you

report ?


Yes, that perfectly solves the issues Lazarus developers and users face. 
I am OK with this solution as well. Thanks!


Ondrej

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] TEncoding.Default and default encoding for TStrings.LoadFrom*()

[Ondrej Pokorny]

On 26.12.2019 23:42, Marco van de Voort wrote:

Op 12/26/2019 om 9:12 PM schreef Ondrej Pokorny:


In Delphi TEncoding.ANSI and TEncoding.Default are actually 
different. See:
http://docwiki.embarcadero.com/Libraries/Rio/en/System.SysUtils.TEncoding.Default 

http://docwiki.embarcadero.com/Libraries/Rio/en/System.SysUtils.TEncoding.ANSI 



On Windows, they are equal but on POSIX they are different: 
TEncoding.Default is UTF-8 but TEncoding.ANSI is the code page from 
CFLocaleGetIdentifier.



And in FPC it is exactly the same,


No, it is not. In FPC:
class function TEncoding.GetDefault: TEncoding;
begin
  Result := GetANSI;
end;

For the meaning of TEncoding.Default and TEncoding.ANSI in Delphi see 
the docs above.




BUT Lazarus overrides default with UTF8 on Windows.


Yes, it does it since r61976 - so only recently. And it is a very 
questionable commit because

a) is not Delphi compatible
b) breaks OS-ANSI calls
c) breaks ANSI FPC code

It must either be reverted or we need some high-level method to get the 
OS-ANSI codepage without this override.




As you can see that is NOT compatible with Delphi above.
Yes, and I am against r61976 - but because r61976 overrides 
TEncoding.ANSI to UTF-8 on Windows. IMO TEncoding.Default should be 
UTF-8 in Lazarus even on Windows (whereas TEncoding.ANSI should stay 
OS-ANSI) - I try to explain again why this actually fits very well into 
the Delphi/FPC encoding concept. I will now talk only about Windows for 
simplicity (because the ANSI concept is most important on Windows):


Delphi doesn't know the DefaultSystemEncoding concept that FPC has. The 
default AnsiString encoding in Delphi is always OS-ANSI (CP_ACP). 
Therefore it makes perfect sense to have TEncoding.Default to point to 
the default AnsiString encoding that is OS-ANSI encoding in Delphi.


FPC, on the contrary, overrides the CP_ACP value with 
DefaultSystemEncoding. So the default AnsiString encoding is not OS-ANSI 
but DefaultSystemEncoding. Therefore, again, it makes perfect sense to 
have TEncoding.Default to point to the default AnsiString encoding that 
is DefaultSystemEncoding in FPC.


Onrej
___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] TEncoding.Default and default encoding for TStrings.LoadFrom*()

[Ondrej Pokorny]

On 27.12.2019 0:19, Michael Van Canneyt wrote:

On Thu, 26 Dec 2019, Ondrej Pokorny wrote:


On 26.12.2019 19:29, Michael Van Canneyt wrote:
So no, I don't think these need to be changed/merged. What IMO can 
be discussed is
which of these 2 need to be used as the default codepage in other 
code. It

should then resolve the problems that appear, I think.


That would be possible as well. But still please reconsider it:
One reason: just from the convention - the default codepage to use 
should be TEncoding.Default. That is intuitive.


Second reason: Now we have TEncoding.ANSI = TEncoding.Default. 2 
equal properties. And another FPC-only property 
TEncoding.SystemEncoding. That means 3 properties for 2 values.


As far as I know, TEncoding.ANSI = CP_ACP.


This is indeed not correct. See 
https://wiki.freepascal.org/FPC_Unicode_support :
CP_ACP: this value represents the currently set "default system code 
page". See #Code page settings for more information.


The code for it is in sysos.inc:
function TranslatePlaceholderCP(cp: TSystemCodePage): TSystemCodePage; 
{$ifdef SYSTEMINLINE}inline;{$endif}

begin
  TranslatePlaceholderCP:=cp;
  case cp of
    CP_OEMCP:
  TranslatePlaceholderCP:=GetOEMCP;
    CP_ACP:
  TranslatePlaceholderCP:=DefaultSystemCodePage;
  end;
end;

Whereas TEncoding.ANSI is the WIN-ANSI OS encoding:

class function TEncoding.GetANSI: TEncoding;
// ...
    FStandardEncodings[seAnsi] := 
TMBCSEncoding.Create(widestringmanager.GetStandardCodePageProc(scpAnsi))


and
  TStandardCodePageEnum = (
    scpAnsi, // system Ansi code page (GetACP on windows)

- as you can see the CP_ACP value does not correspond with the GetACP 
WinAPI call result. (But this is wanted as documented in 
https://wiki.freepascal.org/FPC_Unicode_support ).


Why should this equal TEncoding.Default ? 


sysencoding.inc:

class function TEncoding.GetDefault: TEncoding;
begin
  Result := GetANSI;
end;


I think  TEncoding.Default  = CP_UTF8 on linux ?


Yes, in FPC this is correct. Also TEncoding.ANSI =CP_UTF8 on linux in FPC.



The main problem I see is that there is the system (OS) encoding, and the
encoding specified by DefaultSystemCodePage.

These do not necessarily agree. So it makes sense to have 2 
TEncodings: one

for the system encoding, one for the DefaultSystemCodePage variable. They
will not be equal.

If they were, then the DefaultSystemCodePage variable makes no sense 
whatever.


Yes, indeed. Therefore I suggested
* TEncoding.Default for the DefaultSystemCodePage variable
and
* TEncoding.ANSI for the system encoding.

Currently we have
* TEncoding.SystemEncoding for the DefaultSystemCodePage variable
and
* both TEncoding.ANSI and TEncoding.Default for the system encoding. 
(TEncoding.ANSI and TEncoding.Default are equal in FPC.)


Ondrej

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] TEncoding.Default and default encoding for TStrings.LoadFrom*()

[Ondrej Pokorny]

On 26.12.2019 19:29, Michael Van Canneyt wrote:
So no, I don't think these need to be changed/merged. What IMO can be 
discussed is
which of these 2 need to be used as the default codepage in other 
code. It

should then resolve the problems that appear, I think.


That would be possible as well. But still please reconsider it:
One reason: just from the convention - the default codepage to use 
should be TEncoding.Default. That is intuitive.
Second reason: Now we have TEncoding.ANSI = TEncoding.Default. 2 equal 
properties. And another FPC-only property TEncoding.SystemEncoding. That 
means 3 properties for 2 values.

---

In Delphi TEncoding.ANSI and TEncoding.Default are actually different. See:
http://docwiki.embarcadero.com/Libraries/Rio/en/System.SysUtils.TEncoding.Default
http://docwiki.embarcadero.com/Libraries/Rio/en/System.SysUtils.TEncoding.ANSI

On Windows, they are equal but on POSIX they are different: 
TEncoding.Default is UTF-8 but TEncoding.ANSI is the code page from 
CFLocaleGetIdentifier.


Read the .NET docs about Encoding.Default:
https://docs.microsoft.com/en-us/dotnet/api/system.text.encoding.default?redirectedfrom=MSDN=netframework-4.8#System_Text_Encoding_Default
on .NET Framework it is ANSI but on .NET Core it is UTF-8 even on Windows.

With all the information from the docs, I am more and more convinced 
that TEncoding.SystemEncoding is superfluous and TEncoding.Default 
should take over its meaning: TEncoding.Default should reflect changes 
in DefaultSystemCodePage. Whereas TEncoding.ANSI should stay a fixed 
ANSI code page. With it there is no need for TEncoding.SystemEncoding.


With this change, in the current Lazarus UTF-8 solution, 
TEncoding.Default will be UTF-8. In the future Unicode and 
Delphi-compatible FPC/Lazarus, TEncoding.Default will get the Delphi 
meaning (ANSI/UTF-8). IMO the concept is very sensible.


---

Btw. you have a bug in:

constructor TStringStream.CreateRaw(const AString: RawByteString);
var
  CP: TSystemCodePage;
begin
  CP:=StringCodePage(AString);
  if (CP=CP_ACP) or (CP=TEncoding.Default.CodePage) then // this line 
is wrong

    begin
    FEncoding:=TEncoding.Default;
    FOwnsEncoding:=False;
    end
  else

In the code above, TEncoding.Default is used if CP=CP_ACP. That is 
currently wrong - the bug perfectly reflects my suggestion for 
TEncoding.Default change. Currently, CP_ACP corresponds with 
DefaultSystemEncoding and thus with TEncoding.SystemEncoding and not 
TEncoding.Default. TEncoding.Default corresponds with ANSI (that is not 
CP_ACP as documented https://wiki.freepascal.org/FPC_Unicode_support ).


The code should be:
if (CP=CP_ACP) or (CP=TEncoding.SystemEncoding.CodePage) then
begin
  FEncoding:=TEncoding.SystemEncoding;
  FOwnsEncoding:=False;
end else
if (CP=TEncoding.Default.CodePage) then
begin
  FEncoding:=TEncoding.Default;
  FOwnsEncoding:=False;
end else
// ...

The current CreateRaw code is correct for my suggestion. As you can see 
you intuitively expected the approach I am suggesting :)


Ondrej

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] TEncoding.Default and default encoding for TStrings.LoadFrom*()

[Ondrej Pokorny]

On 26.12.2019 17:02, Mattias Gaertner via fpc-devel wrote:

On Thu, 26 Dec 2019 16:55:04 +0100
Ondrej Pokorny  wrote:


On 26.12.2019 16:41, Mattias Gaertner via fpc-devel wrote:

On Thu, 26 Dec 2019 16:15:03 +0100
Ondrej Pokorny  wrote:
  

Hello,

a lot of people have a problem with the TStrings.LoadFrom*()
changes when TEncoding support was added.

Currently, the no-encoding overloads of TStrings.LoadFrom*() and
TStrings.SaveTo*() use the TEncoding.Default, which is WIN-ANSI and
not DefaultSystemCodePage.

It seems FPC 3.3.1 does use DefaultSystemCodePage:

class function TEncoding.GetANSI: TEncoding;
begin

  if not Assigned(FStandardEncodings[seAnsi]) then
  begin
// DefaultSystemCodePage can be set to non-ANSI
if Assigned(widestringmanager.GetStandardCodePageProc) then
  FStandardEncodings[seAnsi] :=
TMBCSEncoding.Create(widestringmanager.GetStandardCodePageProc(scpAnsi))
else FStandardEncodings[seAnsi] :=
TMBCSEncoding.Create(DefaultSystemCodePage); ...
end;

Check the code more carefully. It uses DefaultSystemCodePage only
when no widestringmanager is present - which is basically never the
case (at least on win32, Linux, Mac OS).

It uses widestringmanager.GetStandardCodePageProc(scpAnsi) that is
WIN-ANSI on win32 (typically 1250, 1251, 1252 - depending on your OS
language version).

Yes, I just saw it. Bummer.


The comment
// DefaultSystemCodePage can be set to non-ANSI
is misleading and doesn't correspond to both the code and the currently 
desired behavior https://bugs.freepascal.org/view.php?id=32961#c115162


I deleted it.

Ondrej

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] TEncoding.Default and default encoding for TStrings.LoadFrom*()

[Ondrej Pokorny]

On 26.12.2019 16:41, Mattias Gaertner via fpc-devel wrote:

On Thu, 26 Dec 2019 16:15:03 +0100
Ondrej Pokorny  wrote:


Hello,

a lot of people have a problem with the TStrings.LoadFrom*() changes
when TEncoding support was added.

Currently, the no-encoding overloads of TStrings.LoadFrom*() and
TStrings.SaveTo*() use the TEncoding.Default, which is WIN-ANSI and
not DefaultSystemCodePage.

It seems FPC 3.3.1 does use DefaultSystemCodePage:

class function TEncoding.GetANSI: TEncoding;
begin
   
 if not Assigned(FStandardEncodings[seAnsi]) then
 begin
   // DefaultSystemCodePage can be set to non-ANSI
   if Assigned(widestringmanager.GetStandardCodePageProc) then
 FStandardEncodings[seAnsi] := 
TMBCSEncoding.Create(widestringmanager.GetStandardCodePageProc(scpAnsi))
   else
 FStandardEncodings[seAnsi] := 
TMBCSEncoding.Create(DefaultSystemCodePage);
...
end;


Check the code more carefully. It uses DefaultSystemCodePage only when 
no widestringmanager is present - which is basically never the case (at 
least on win32, Linux, Mac OS).


It uses widestringmanager.GetStandardCodePageProc(scpAnsi) that is 
WIN-ANSI on win32 (typically 1250, 1251, 1252 - depending on your OS 
language version).


Ondrej

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

[fpc-devel] TEncoding.Default and default encoding for TStrings.LoadFrom*()

[Ondrej Pokorny]

Hello,

a lot of people have a problem with the TStrings.LoadFrom*() changes 
when TEncoding support was added.


Currently, the no-encoding overloads of TStrings.LoadFrom*() and 
TStrings.SaveTo*() use the TEncoding.Default, which is WIN-ANSI and not 
DefaultSystemCodePage.


Before the changes, the encoding was just ignored and the file was read 
without any codepage support (thus in fact in DefaultSystemCodePage).


A lot of people have a problem with it because in Lazarus they used to 
read UTF-8 text with TStrings.LoadFrom*(). Now the text is broken 
because TStrings.LoadFrom*() load the text in WIN-ANSI.


The Lazarus team came with some kind of solution: they changed the FPC 
internal function widestringmanager.GetStandardCodePageProc to return 
UTF-8 and thus TEncoding.Default and TEncoding.ANSI to be UTF-8. This, 
as a result, broke every ANSI code that needs real WIN-ANSI codepage 
(like the ODBC: https://bugs.freepascal.org/view.php?id=36481 ). (And it 
introduced more Delphi & other Lazarus incompatibilities just for the 
sake of keeping TStrings.LoadFrom*() legacy-Lazarus compatible.)

For me the Lazarus solution is not acceptable.

But other Lazarus team members seem not to be happy with the encoding 
breaking change in FPC TStrings.LoadFrom*() and SaveTo*() and ask for a 
solution.


---

I suggest a compromise (steps):

1.) Keep TEncoding.ANSI always WIN-ANSI and Delphi-compatible. (Don't 
change it to DefaultSystemCodePage in Lazarus.)
2.) Change TEncoding.Default value to current TEncoding.SystemEncoding. 
I.e. TEncoding.Default would correspond to DefaultSystemCodePage and 
CP_ACP. Yes, this will be Delphi-incompatible - but CP_ACP is 
Delphi-incompatible as well (!) - so the incompatibilities are 
consequent here.
3.) Delete TEncoding.SystemEncoding because it is an FPC-only construct, 
it is not needed anymore (because it will become TEncoding.Default) and 
it has not been released in any stable version.


The above steps perfectly correlate with the CP_ACP value corresponding 
to DefaultSystemEncoding. As I wrote before, the CP_ACP is not 
Delphi-compatible either. See 
https://wiki.freepascal.org/FPC_Unicode_support


It makes perfect sense to keep TEncoding consequent with CP_ACP and make 
TEncoding.Default to correspond with DefaultSystemEncoding.


It is also natural: I would expect TEncoding.Default to correspond with 
the default (Ansi)String encoding, which is DefaultSystemEncoding.


Ondrej

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] TFPList / TList Sort with Context parameter

[Ondrej Pokorny]

I needed to get things working so I could not wait for an answer any 
longer. I posted a patch: https://bugs.freepascal.org/view.php?id=35962


Best
Ondrej

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] TFPList / TList Sort with Context parameter

[Ondrej Pokorny]

On 12.08.2019 13:02, Ondrej Pokorny wrote:
If this is not possible (which I think it is not), will a patch be 
accepted that adds these methods to TFPList/TList?:

    procedure Sort(Compare: TListSortComparer_Context);
    procedure Sort(Compare: TListSortComparer_Context; 
SortingAlgorithm: PSortingAlgorithm);

The methods will be:
    procedure Sort(Compare: TListSortComparer_Context; Context: Pointer);
    procedure Sort(Compare: TListSortComparer_Context; Context: 
Pointer; SortingAlgorithm: PSortingAlgorithm);


obviously...

Ondrej
___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

[fpc-devel] TFPList / TList Sort with Context parameter

[Ondrej Pokorny]

Hello!

I see that sortbase.pp has prepared sorting functionality with context 
(=custom parameter) that I need to use. See 
TSortingAlgorithm.PtrListSorter_ContextComparer and 
QuickSort_PtrList_Context.


But I don't know how to use them within TList because TFPList / TList 
doesn't publish any Sort() overloads that use them. Furthermore FList is 
private which means I cannot access it in a derived object and I cannot 
add this functionality for my own list object.


My question is: how can I sort a TList/TFPList with a custom context 
parameter?:


  TList = class(TObject,IFPObserved)
    // ...
    procedure Sort(Compare: TListSortComparer_Context);
  end;

If this is not possible (which I think it is not), will a patch be 
accepted that adds these methods to TFPList/TList?:

    procedure Sort(Compare: TListSortComparer_Context);
    procedure Sort(Compare: TListSortComparer_Context; 
SortingAlgorithm: PSortingAlgorithm);


Best
Ondrej

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] [Suggestion] Enumeration range-check intrinsic

[Ondrej Pokorny]

On 13.07.2019 21:26, Michael Van Canneyt wrote:
I think all sides have now been reviewed to the point of boring or 
annoying each other almost to death, and we finally need to decide on 
whether the patch is applied, and if so, which parts of it.


That would be great. I have been waiting for some kind of decision or at 
least some statement from the compiler team for more than a year. Are 
there any news regarding the decision progress?


Ondrej

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] Evaluation order in expressions

[Ondrej Pokorny]

On 30.07.2019 09:46, Michael Van Canneyt wrote:

I think they updated the documentation meanwhile:

http://docwiki.embarcadero.com/RADStudio/Rio/en/Procedures_and_Functions_(Delphi)#Calling_Conventions 



So you can see it depends on the calling convention.


Thanks, I wasn't aware of it!

Ondrej

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] Evaluation order in expressions

[Ondrej Pokorny]

On 30.07.2019 08:55, Jonas Maebe wrote:
Delphi evaluates e.g. all parameters to subroutine calls from left to 
right, which FPC does not guarantee.


If I am not mistaken Delphi doesn't guarantee this either. I remember I 
"learned it by doing" quite some time ago and I don't think I used FPC 
at that point, so it must have been Delphi.


A quick Google search revealed https://stackoverflow.com/a/11010685 - 
yes, when I run the code from the answer in Delphi 32bit the right 
"SideEffect(2)" call is executed before the left "SideEffect(1)" call.


Ondrej

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] Evaluation order in expressions

[Ondrej Pokorny]

On 30.07.2019 08:33, Michael Van Canneyt wrote:

The documentation has already been fixed in this regard, there was a
bugreport about it not so long ago.


Great, thank you. Is there some online version of the latest 
documentation trunk (nightly build)?


Ondrej

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] Evaluation order in expressions

[Ondrej Pokorny]

On 30.07.2019 08:23, Ondrej Pokorny wrote:
/Remark: The order in which expressions of the same precedence are 
evaluated is not guaranteed to be left-to-right./


Is this valid for boolean expressions, too? I use the following 
construct with {$BOOLEVAL OFF}:

Result := Assigned(MyObject) and (MyObject.Value>5);

A moment of relief. The FPC compiler itself depends on the left-to-right 
evaluation. E.g. compiler/aoptbase.pas, line 284:


    while assigned(p1) and assigned(p2) and GetNextInstruction(p1,p1) 
and (p1<>p2) do



So is the documentation wrong or are the compiler (and also my own) 
sources flawed?


Best
Ondrej

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

[fpc-devel] Evaluation order in expressions

[Ondrej Pokorny]

Hello,

can somebody comment on the Remark from 
https://www.freepascal.org/docs-html/ref/refch12.html ?
/Remark: The order in which expressions of the same precedence are 
evaluated is not guaranteed to be left-to-right./


Is this valid for boolean expressions, too? I use the following 
construct with {$BOOLEVAL OFF}:

Result := Assigned(MyObject) and (MyObject.Value>5);
The remark above suggests that I cannot use it because 
(MyObject.Value>5) may be evaluated before Assigned(MyObject) if the 
optimizer decides so.


On the other hand, the information in 
https://www.freepascal.org/docs-html/current/prog/progsu4.html#x11-11.2.4 
suggests the opposite:
/So, in the following example, the function Bofu, which has a boolean 
result, will never get called.//

//If False and Bofu then/

Delphi documentation also explicitly states that the expressions are 
evaluated in left to right order: 
http://docwiki.embarcadero.com/RADStudio/Rio/en/Boolean_short-circuit_evaluation_(Delphi_compiler_directive)


(Well it is no surprise because 
https://www.freepascal.org/docs-html/ref/refch12.html states that "/This 
behaviour is distinctly different from Delphi or Turbo Pascal./")


Ondrej

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] Cannot build fpc trunk on MacOS

[Ondrej Pokorny]

On 28.07.2019 13:15, Jonas Maebe wrote:

On 28/07/2019 10:41, Ondrej Pokorny wrote:

I wanted to try the latest FPC/Lazarus on Mac OS and I got the following
error when building trunk with FPC 3.0.4:

Sorry, fixed.


I can build FPC trunk on MacOS now, thank you!

Ondrej

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

[fpc-devel] Cannot build fpc trunk on MacOS

[Ondrej Pokorny]

Hello,

I wanted to try the latest FPC/Lazarus on Mac OS and I got the following 
error when building trunk with FPC 3.0.4:


Installing package chm
Installation package chm for target x86_64-darwin succeeded
Start compiling package cocoaint for target x86_64-darwin.
   Compiling cocoaint/BuildUnit_cocoaint.pp
Compiled package cocoaint
Installing package cocoaint
The installer encountered the following error:
Unable to open file "cocoaint/units/x86_64-darwin/CoreGraphics.ppu"
make[3]: *** [distinstall] Error 1
make[2]: *** [packages_distinstall] Error 2
make[1]: *** [installother] Error 2
make: *** [installall] Error 2

I compile with this script:

make clean all OPT="-gl -O-"
make install INSTALL_PREFIX=~/software/fpc/trunk_exp PP=/usr/local/bin/fpc

I downloaded and installed FPC 3.0.4 from 
https://www.freepascal.org/download.html. The version is as follows:


Free Pascal Compiler version 3.0.4 [2018/09/30] for x86_64
Copyright (c) 1993-2017 by Florian Klaempfl and others


Anybody knows what's going on?

Thanks
Ondrej

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] Internal error 200510032 on SynEdit package

[Ondrej Pokorny]

On 23.07.2019 16:05, denisgolovan wrote:

Anybody knows whats going on and why it stopped working? (I am on
Windows and use the 32bit compiler. Today's FPC trunk, today's Lazarus
trunk.)

Same story.
See https://bugs.freepascal.org/view.php?id=35877


Thanks!
Ondrej

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

[fpc-devel] Internal error 200510032 on SynEdit package

[Ondrej Pokorny]

Hello,

I updated trunk now and wanted to test the attributes support - and now 
I get Internal error 200510032 when compiling the SynEdit package. (See 
attached screenshot.)


synedithighlighterxmlbase.pas(236,31) Error: Internal error 200510032

Rebuild clean doesn't help. I can go over the error when I change this line:

  Result := TSynXmlRangeInfo((ItemPointer[Index] + FItemOffset)^);

to

var
  P: Pointer;
// 
  P := ItemPointer[Index] + FItemOffset;
  Result := TSynXmlRangeInfo(P^);

Anybody knows whats going on and why it stopped working? (I am on 
Windows and use the 32bit compiler. Today's FPC trunk, today's Lazarus 
trunk.)


Best
Ondrej

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] Property attributes support

[Ondrej Pokorny]

On 20.07.2019 13:56, Sven Barth via fpc-devel wrote:

Here: https://wiki.freepascal.org/Custom_Attributes


Thank you!


The old Wiki page should propably be updated to reflect the current 
state of affairs...


I deleted the old information (nearly everything) and put a link to the 
new Wiki page there.



You can also take a look at Delphi tutorials. The only difference is 
that we provide access through the TypInfo unit as well while Delphi 
officially only supports the Rtti unit.


I am familiar with Delphi attributes - I didn't expect the FPC 
attributes to be Delphi-compatible. Thanks again!


Ondrej

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

[fpc-devel] Property attributes support

[Ondrej Pokorny]

Hello!

I saw that Sven committed property attributes support in trunk. Thank 
you for this!


Is there any documentation how to use them? (Syntax, reading information 
during runtime etc?) I only found very old information:

https://wiki.freepascal.org/Property_attributes
http://free-pascal-general.1045716.n5.nabble.com/Property-attributes-td5718442.html

Best
Ondrej

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] [Suggestion] Enumeration range-check intrinsic

[Ondrej Pokorny]

On 13.07.2019 13:47, J. Gareth Moreton wrote:
Here lies the problem - if you have a enumeration stored in a byte 
field, say, in a file, and there's a chance it's been corrupted, 
there's no reliable way to detect that, since even typecasting may not 
be reliable, and once again I'm implying that enums are unsafe to use, 
but right now, what else can one do?


Don't forget about my second suggestion: add internal range checking for 
enumeration values on specific operations: 
https://lists.freepascal.org/pipermail/fpc-devel/2019-July/041499.html


The "BUT (big BUT)" section in the 2nd half of the email. I copy it here 
with one small update:


BUT (big BUT): I suggest to do an enumeration/subrange validity check on 
the compiler side when range checking is used. This is similar to the 
current fpc_check_object intrinsic.


When range checking is used, this check should be injected (at least) 
before every case-of and the new is/as operators if they are replaced 
with true. When range checking is disabled, the check should not be 
injected at all. Examples for range checking:


1.) case MyEnumValue of // <<< here an enumeration validity check should 
be injected
2.) IsValid := SomeValue is TEnumValue; //  <<< here an enumeration 
validity check should be injected if SomeValue is an enumeration type. 
(It doesn't make sense for normal ordinal values like Integer because 
they are valid in the whole stored range.)

That would catch the
IsValid := MyEnumValue is TEnumValue;
check - it would be replaced by a TRUE constant but there would still be 
a range check injected that would raise an error in case MyEnumValueis 
not valid.


IMO such a solution is acceptable for both sides:
1.) Jonas' point: The clarity of the language will stay untouched 
because the new validity check will be only informative and used only 
for debugging without any assurance of correctness. (You see the 
difference: MyEnumValue is TEnumValue will raise an exception for an 
invalid value and not return false! False will be returned only for 
(MyInt is TEnumValue) if MyInt is ot of the range of TEnumValue.)


2.) My/J.Gareth's point: We will get an information about invalid enum 
values (range check error) instead of "undefined behavior" when 
debugging - and we
will be able to find code where invalid enum values can occur and fix 
the code where these values were assigned. Although this information may 
not be (theoretically) absolutely accurate (the same as fpc_check_object 
is not absolutely accurate), it will still be pretty accurate and it 
will be a big help during the debugging.


Ondrej

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] [Suggestion] Enumeration range-check intrinsic

[Ondrej Pokorny]

On 13.07.2019 14:32, Michael Van Canneyt wrote:

I yesterday checked a database project I have. I counted 25 enumerateds
which are stored in the database. Without fail, for each type there is 
somewhere code in a type helper:


if (MyDatabaseValue>=0) and (MyDatabaseValue<=Ord(High(TMyENum))) then
  MyENumValue:=TEnum(MyDatabaseValue))
else
  EConvertError.Create('Invalid value for TMyEnum : %d',[MyDatabaseValue])


I do exactly the same - check the low/high bounds in a type helper :) 
Yes, and I am tired of typing it as well :)




That the case where you would write

if MyEnumValue is TMyEnumValue then

can 'by default' be optimized away because as far as the compiler is
concerned this is a tautology, is for me a perfectly acceptable and
defendable point of view.


I agree.



Even to such a degree that the compiler would
simply reject your code by not allowing is/at on a variable of the type
itself. Also defendable.


No - not this one. Don't forget about e.g. generics:
function Test(MyValue: T): Boolean;
begin
  Result := MyValue is TMyEnum;
end;
would compile for all enumeration types but not for MyValue:TMyEnum. 
That doesn't make sense.


Better to have a compiler hint/warning when such a construct is found 
and optimized. This perfectly corresponds with the fact that also 
constructs like "SmallIntValue is Integer" can be optimized away but 
must be allowed.


Ondrej

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] [Suggestion] Enumeration range-check intrinsic

[Ondrej Pokorny]

On 11.07.2019 11:58, J. Gareth Moreton wrote:


Done!


Thank you for your work!

BTW, your note in the issue report is very misleading: 
https://bugs.freepascal.org/view.php?id=33603#c117162.


> The idea behind "is" and "as" is that they are the sole exception to 
the rule and will return False and raise an error respectively if the 
enum is out of bounds.


IS/AS are not an exception of the rule - after a long discussion we 
settled down that if IS/AS is applied on a value of the same enumeration 
type:


var
  MyEnumValue: TEnumValue;
begin
  if MyEnumValue is TEnumValue then
// ...
end;

the compiler will expect that the result is always true. (It doesn't 
have to check for invalid values.)


When I read the following sentence:
> If the value falls within a hole, then it will pass - currently only 
in Delphi mode though. FPC mode won't allow it.


I think you mixed up "normal enums" and "enums with holes" and so the 
note in the issue report doesn't make any sense.




The conclusion is:
1.) IS/AS works reliably only on valid values on the left side of the 
operator. (Like any other function/intrinsic/operator in FPC.)
2.) If an invalid value is to be expected, IS/AS must be executed on an 
ordinal value (Integer, SmallInt, ...) before assigning it to the 
enumeration value. It cannot be executed on the resulting enumeration 
value to check its validity.
3.) IS/AS can be used also on "enumerations with holes". In this respect 
only the low/high bounds are checked, not the holes inside the valid range.

4.) IS/AS can be also used on ordinal types:
 - 4a.) (MyInt64Value is Integer) is a reasonable check if MyInt64Value 
is in Integer range.
 - 4b.) (MyIntegerValue is Int64) is not reasonable (Integer falls 
always into Int64 range) and will always return True and can/will be 
optimized by the compiler to return a True constant.


Ondrej

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] [Suggestion] Enumeration range-check intrinsic

[Ondrej Pokorny]

On 08.07.2019 17:19, J. Gareth Moreton wrote:

I disabled it after Pierre pointed out
that you couldn't use Pred or Succ with
them, so wanted to be consistent,
especially as programming them to return
the expected value of False of a value
falls within a hole is not simple. Still,
it's easy enough to re-enable.

Gareth aka. Kit


There has been a much bigger discussion about it than the one post from 
Pierre.


Ondrej


___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] [Suggestion] Enumeration range-check intrinsic

[Ondrej Pokorny]

On 08.07.2019 02:37, J. Gareth Moreton wrote:


Uploaded the latest patch (AS-IS-enum-07.patch) over here 
 and also a test 
project that evaluates the "is" operator between different ordinal types.


I though I convinced you not to disable the feature on enums with holes 
even in OBJFPC mode...


Ondrej


___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] [Suggestion] Enumeration range-check intrinsic

[Ondrej Pokorny]

On 08.07.2019 12:39, Giuliano Colla wrote:

Il 07/07/2019 07:33, J. Gareth Moreton ha scritto:

In the meantime, I'm working on making
"as" and "is" work with ordinal types as
well as enumerations, although currently
some headaches occur if the right-hand
side is larger than the CPU word size
(e.g. Int64 on i386). I'll upload the
patch to the issue once it's working
properly.


Sorry for jumping in without having followed the full discussion, but 
I wonder if "as" and "is" are the best constructs to implement range 
checking. Wouldn't "in" be much more appropriate?


Sort of:

if value in enumvar then DoThat else IgnoreIt;


The IN operator
1.) expects a value on the right side, not a type.
2.) can be overloaded: 
https://www.freepascal.org/docs-html/ref/refse104.html#x213-23500015.6 . 
IS/AS cannot.


Ondrej

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] [] property overloads

[Ondrej Pokorny]

On 06.07.2019 22:12, Ryan Joseph wrote:

On Jul 6, 2019, at 4:09 PM, Ondrej Pokorny  wrote:

On 06.07.2019 20:03, Ryan Joseph wrote:

Yeah that’s correct.

Here’s my bug report:

https://bugs.freepascal.org/view.php?id=35809

Did you just create a duplicate bug report to the one I linked in this thread 
before ( https://lists.freepascal.org/pipermail/fpc-devel/2019-July/041350.html 
) or did you have something different in mind with your bug report?

I’m sorry, I didn’t know about your report.


I though you checked it when you answered upon it. But no problem - it 
happens quite often that people answer my posts without actually reading 
them. On the other hand it is quite exhausting to keep repeating myself ;)



I see Sven says it is indeed a bug and will be fixed.


Yes, I reported it as bug indeed - the notes about existing property 
overload support was a little bit ironic (just a little bit ;) ).


Ondrej

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] [] property overloads

[Ondrej Pokorny]

On 06.07.2019 20:03, Ryan Joseph wrote:

Yeah that’s correct.

Here’s my bug report:

https://bugs.freepascal.org/view.php?id=35809


Did you just create a duplicate bug report to the one I linked in this 
thread before ( 
https://lists.freepascal.org/pipermail/fpc-devel/2019-July/041350.html ) 
or did you have something different in mind with your bug report?


Ondrej

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] [Suggestion] Enumeration range-check intrinsic

[Ondrej Pokorny]

On 05.07.2019 22:44, J. Gareth Moreton wrote:


In the meantime, I've extended your AS/IS patch over here 
 to create efficient 
code for x86 platforms, although currently it only does a range check 
and won't correctly handle enumerations with holes.  If non-contiguous 
enumerations are going to be allowed, we'll need to design an 
algorithm that can cover these holes with the smallest number of 
Boolean conditions.


IMO handling holes in enumerations is just a pure waste of time and 
energy. Delphi documents them as valid and the FPC compiler handles them 
as valid as well. IMO it is just an unnecessary extra added limitation 
to disable IS/AS on them. Furthermore they are used mainly for very 
specific needs, so are very marginal. The effort to make this work just 
doesn't pay off.


Btw. you get a "project1.lpr(7,10) Warning: Case statement does not 
handle all possible cases" in the program below anyway:


program Project1;
type
  THoleEnum = (two=2, four=4);
var
  E: THoleEnum;
begin
  case E of
    two: ;
    four: ;
  end;
end.

What will you do now, disable case-of for enumerations with holes 
because the compiler cannot handle the holes correctly in the strict FPC 
POV way?


Ondrej

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] [Suggestion] Enumeration range-check intrinsic

[Ondrej Pokorny]

On 06.07.2019 02:10, Martok wrote:

In that case, unlike what was previously said in this thread, this should simply
work:

generic function TStreamHelper.ReadEnum(out aEnum: T): Boolean;
begin
   if Read(aEnum, SizeOf(aEnum)) <> SizeOf(aEnum) then
 Exit(False)
   else begin
 Result := (Ord(aEnum) >= Ord(Low(T))) and (Ord(aEnum) <= Ord(High(T)));
   end;
end;


Ord(aEnum) for invalid enumeration values is undefined ;)

Ondrej

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] [Suggestion] Enumeration range-check intrinsic

[Ondrej Pokorny]

On 05.07.2019 16:51, Pierre Muller wrote:

This means that using pred() or succ() intrinsics on enumerated types with
holes will generate a Compile Time Error.

   To be consistent, I would propose that we also generate
a Compile Time Error if 'is' or 'as' keyword is used on such a type,
unless there is a code that really check that the value is not in
one of the holes ...


I have to strongly disagree here. We need the IS/AS operators to prevent 
undefined behavior. And if undefined behavior is only outside the 
low/high bounds of the enumeration values and not within the holes - 
then the IS/AS that checks only the bounds is very consequent indeed. 
It's also consequent with the case-of statement for an enum with holes. 
(See the thread I linked before.)


Ondrej

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] [Suggestion] Enumeration range-check intrinsic

[Ondrej Pokorny]

On 05.07.2019 16:51, Pierre Muller wrote:

Just one point from current compiler implementation:

in trunk/fpcsrc/compiler/ninl.pas (around line 3180)

   in_pred_x,
   in_succ_x:
 begin
set_varstate(left,vs_read,[vsf_must_be_valid]);
resultdef:=left.resultdef;
if is_ordinal(resultdef) or is_typeparam(resultdef) then
  begin
if (resultdef.typ=enumdef) and
   (tenumdef(resultdef).has_jumps) and
   not(m_delphi in current_settings.modeswitches) and
   not(nf_internal in flags) then
  
CGMessage(type_e_succ_and_pred_enums_with_assign_not_possible);
  end


This means that using pred() or succ() intrinsics on enumerated types with
holes will generate a Compile Time Error.


Only for non-Delphi modes, see: not(m_delphi in 
current_settings.modeswitches)




   To be consistent, I would propose that we also generate
a Compile Time Error if 'is' or 'as' keyword is used on such a type,
unless there is a code that really check that the value is not in
one of the holes ...


See this thread: 
https://lists.freepascal.org/pipermail/fpc-devel/2019-May/040865.html


Ondrej

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] [Suggestion] Enumeration range-check intrinsic

[Ondrej Pokorny]

On 05.07.2019 14:58, Martok wrote:

Am 05.07.2019 um 12:20 schrieb Ondrej Pokorny:

Anything that even looks like a subrange cannot be used for these interfaces.

You may use enumerations for these interfaces - you may just not fill
them directly but through an integer variable in between when reading
from outside.

Then you're not using them *in* the interface ;-)


For records you could use such a workaround:

program Project1;
{$mode objfpc}
{$modeswitch advancedrecords}
type
  TMyEnum = (one, two, three);
  TMyRecord = record
  public
    function IsValid: Boolean;
  public
    case Boolean of
  False: ( // real values
    Enum: TMyEnum;
  );
  True: ( // used for interface
    EnumIntf: Integer;
    {$IF SizeOf(Integer)<>SizeOf(TMyEnum)}
  {$ERROR Size mismatch}
    {$ENDIF}
  );
  end;
{ TMyRecord }
function TMyRecord.IsValid: Boolean;
begin
  Result := EnumIntf is TMyEnum;
end;

var
  R: TMyRecord;
begin
  // fill R with data ...
  Randomize;
  FillChar(R, SizeOf(R), Random(High(Byte)));
  // check validity
  if R.IsValid then
    Writeln('R is valid')
  else
    Writeln('R is not valid');
  Writeln(R.EnumIntf);
  ReadLn;
end.

Best
Ondrej

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] [Suggestion] Enumeration range-check intrinsic

[Ondrej Pokorny]

On 05.07.2019 14:58, Martok wrote:

Am 05.07.2019 um 12:20 schrieb Ondrej Pokorny:

Anything that even looks like a subrange cannot be used for these interfaces.

You may use enumerations for these interfaces - you may just not fill
them directly but through an integer variable in between when reading
from outside.

Then you're not using them *in* the interface ;-)

Agreed. Ondrej
___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] [Suggestion] Enumeration range-check intrinsic

[Ondrej Pokorny]

On 05.07.2019 11:08, Martok wrote:

Am 05.07.2019 um 02:52 schrieb Michael Van Canneyt:

With this sentence you forbid storing or communicating enumerated values in any 
way:
file, database, over network. It can be used only in a computer program and 
never
leave the context of the running program under any form. Because as soon as
it is somehow communicated, there is a chance it becomes invalid in return
communication.

This is the conclusion from Jonas' arguments, which he explicitly confirmed when
we had the discussion two+ years ago.

You're also missing another important type of 'communication': calling non-FPC
APIs or even just FPC libraries of possibly different version (see: Run-Time
Packages!)

Anything that even looks like a subrange cannot be used for these interfaces.


You may use enumerations for these interfaces - you may just not fill 
them directly but through an integer variable in between when reading 
from outside.


IMO this is not a problem - most modern interfaces work like this 
anyway. E.g. for the database you have the TField.AsInteger:


if MyField.AsInteger is TMyEnum then
  MyEnumValue := TMyEnum(MyField.AsInteger)
else
  // do something in case of invalid value

The above code will work perfectly and safely.

For REST or SOAP APIs etc. you have a JSON or XML reader that again 
returns an Extended (JSON) or string (XML) so again this is no problem 
at all.


Only the old-fashioned direct reads (from files, streams etc.) must be 
done with an extra integer variable in between. But that is safer anyway 
because e.g. if you add new enumeration values the SizeOf(TMyEnum) may 
increase and then you have a big problem anyway.


So all-in-all the 2 features solve the problem:
1.) IS/AS operators. They will save some typing (check for low-high bounds).
2.) Additional compiler built-in enumeration value checks when 
range-checking is on. They will help to find places where an invalid 
enumeration value may occur and so the program code can be fixed during 
debugging.


Ondrej

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] [Suggestion] Enumeration range-check intrinsic

[Ondrej Pokorny]

On 05.07.2019 11:33, Michael Van Canneyt wrote:

On Fri, 5 Jul 2019, Ondrej Pokorny wrote:

On 05.07.2019 10:52, Sven Barth via fpc-devel wrote:
Michael Van Canneyt <mailto:mich...@freepascal.org>> schrieb am Fr., 5. Juli 2019, 10:47:


    If memory serves well, FPC introduced this (before delphi) to be
    able to import C enums with assigned values.


Correct. That is also why we allow both "=" and ":=" for the value 
assignments. Because Delphi only supports one of the two and we 
picked the other back then. ;)


Ignorant Delphi core developers have never checked how FPC does 
things before implementing their own solution to keep compatibility 
;) And then they added ARC, then they removed ARC, they then added 
0-based strings, then they killed the product (AppMethod) they made 
it for... Unfortunately the 0-based string features made it into FPC 
in TStringHelper as well...


TStringHelper is for delphi compatibility, so there is no choice.


I know, I am not blaming you (or anybody else) for doing so.

Still, in my opinion the 0-based string feature is the best candidate 
for knowingly breaking Delphi compatibility. The same as the core team 
communicated that the inline var declaration from Delphi 10.3 is not 
welcome in FPC.




All in all, I am still of the opinion that the whole SysUtils unit is a
monster that should be taken apart and its functionality spread over 
various units: StrUtils, DateUtils, IOUtils, Exceptions, Helpers.


Doing so would also release us from the need to be Delphi compatible 
in those

units. Alas, this is meanwhile a huge undertaking.


Yes, that is actually a good compromise: to move the 
Delphi-compatibility TStringHelper into a different unit (e.g. 
DelphiHelpers.pas) and offer a similar helper with 1-based strings in 
Helpers.pas. Yes, there is too much to do :(


Ondrej


___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] [Suggestion] Enumeration range-check intrinsic

[Ondrej Pokorny]

On 05.07.2019 10:52, Sven Barth via fpc-devel wrote:
Michael Van Canneyt > schrieb am Fr., 5. Juli 2019, 10:47:


If memory serves well, FPC introduced this (before delphi) to be
able to import C enums with assigned values.


Correct. That is also why we allow both "=" and ":=" for the value 
assignments. Because Delphi only supports one of the two and we picked 
the other back then. ;)


Ignorant Delphi core developers have never checked how FPC does things 
before implementing their own solution to keep compatibility ;) And then 
they added ARC, then they removed ARC, they then added 0-based strings, 
then they killed the product (AppMethod) they made it for... 
Unfortunately the 0-based string features made it into FPC in 
TStringHelper as well...


Ondrej

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] [Suggestion] Enumeration range-check intrinsic

[Ondrej Pokorny]

On 05.07.2019 09:57, Michael Van Canneyt wrote:

On Fri, 5 Jul 2019, Ondrej Pokorny wrote:

On 05.07.2019 09:04, Michael Van Canneyt wrote:
In this, I would definitely exclude enumerateds that have explicitly 
assigned

values: str does not handle them, getenumename etc. also do not work:
They are in effect simply integer constants. (if I had my way I would
even remove them from the language).

So using the above construct on such an enum can lead to a compiler 
error,
because the compiler cannot check anyway: better a clear error than 
undefined

behaviour.


On the contrary, the compiler can check enums with holes with no 
problem at all. They have valid low/end boundaries. And the holes in 
between are valid values as well.


Eh ? Not in my book, if the compiler allows that it is simply wrong IMO.

An enumerated enumerates the allowed values: no holes allowed.

What you describe, I would name a range, not an enumeration.

So there is no actual need or benefit to disable this feature for 
enums with holes.


See above.

The way I read jonas' argument:

TMyEnum = (one, two=3,three);

A:=TMyEnum(1) ;

will pass unchallenged.

I don't see the point of allowing this. for me, "A" has an invalid value.

Note: This is my opinion, it may be at odds with what the compiler 
actually

implements :)


Read: 
http://docwiki.embarcadero.com/RADStudio/Rio/en/Simple_Types_(Delphi)#Enumerated_Types_with_Explicitly_Assigned_Ordinality




+++

Btw. why nobody commented on my latest suggestion in 
https://lists.freepascal.org/pipermail/fpc-devel/2019-July/041499.html ?

IMO this is a valid solution for both sides.


What part exactly ? The additional range check for a case in case of 
range checking ?


Yes.


I don't see why you need it, since in case range checking is enabled, 
the assignment of the variable should already have ensured that there 
are only valid values in it.


The compiler can never catch all kinds of assignments and do range 
checking on them (e.g. FillChar, Read, pointer-writes, etc. etc.). So a 
range check when accessing a value is needed.


It's the same reasons FPC has fpc_check_object and falls into the same 
philosophy.


If I take your statement "in case range checking is enabled, the 
assignment of the variable should already have ensured that there are 
only valid values in it." to the extreme, it implies that the creation 
of an object with a subrange value always ends up with a range check error:


program Project1;
type
  TMyClass = class
  private
    FSubRange: 2..4;
  end;
  TMyRecord = record
    SubRange: 2..4;
  end;
  PMyRecord = ^TMyRecord;
var
  C: TMyClass;
  R: PMyRecord;
begin
  C := TMyClass.Create; // << range check error here
  R := AllocMem(SizeOf(TMyRecord)); // << range check error here
end.

Ondrej

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] [Suggestion] Enumeration range-check intrinsic

[Ondrej Pokorny]

On 05.07.2019 09:04, Michael Van Canneyt wrote:
In this, I would definitely exclude enumerateds that have explicitly 
assigned

values: str does not handle them, getenumename etc. also do not work:
They are in effect simply integer constants. (if I had my way I would
even remove them from the language).

So using the above construct on such an enum can lead to a compiler 
error,
because the compiler cannot check anyway: better a clear error than 
undefined

behaviour.


On the contrary, the compiler can check enums with holes with no problem 
at all. They have valid low/end boundaries. And the holes in between are 
valid values as well, even if they don't have names assigned. (E.g. the 
warnings in case-of are enabled for enums with holes as well!)


The 02.patch and 05.patch in 
https://bugs.freepascal.org/view.php?id=33603 handle enums with holes 
correctly. (Only the 03.patch disabled IS/AS for holes with enums in 
OBJFPC mode as a complementary feature, but I removed it consequently.)


See Jonas' answer about enums with holes:
https://lists.freepascal.org/pipermail/fpc-devel/2019-May/040868.html

Citation:
> Yet, it still does not answer my question if the holes belong to the
> valid enumeration values from the FPC point-of-view or not.

As far as range checking and undefined behaviour is concerned, they do.
I.e., you won't get undefined behaviour by assigning any value between
low(enum)..high(enum) to them.

+++

So there is no actual need or benefit to disable this feature for enums 
with holes.


+++

Btw. why nobody commented on my latest suggestion in 
https://lists.freepascal.org/pipermail/fpc-devel/2019-July/041499.html ?

IMO this is a valid solution for both sides.

Ondrej

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] [Suggestion] Enumeration range-check intrinsic

[Ondrej Pokorny]

On 04.07.2019 21:20, Jonas Maebe wrote:

On 03/07/2019 09:26, Ondrej Pokorny wrote:

On 02.07.2019 23:34, Jonas Maebe wrote:

Invalid data means undefined behaviour, always. "is" is not a special
case that is immune to this.

Don't you really see the need to handle invalid data with a /defined/
behavior?

My point is that is impossible to do so, so trying to do it in a way
that works in some/most cases, is much more dangerous than categorically
refusing to try to do it, as it creates a false sense of security.

For example:

{$mode objfpc}
type
   tc = class
 a: array[1..2000] of longint;
   end;

   tcc = class of tc;

   trec = record
 p: pointer;
   end;
   prec = ^trec;

var
   c: tc;
   p: prec;
   cc: tcc;
begin
   new(p);
   cc:=tc;
   p^.p:=pointer(cc);
   c:=tc(p);
   writeln(c is tc);
end.

This will print true, even though it's obviously bogus.


Enum and subrange types /can/ store invalid data and they /do/
store invalid data.

The same goes for classes, and by extension for any pointer type.


You have such a validity check for objects in FPC on the compiler side: 
there are compiler intrinsics fpc_check_object & fpc_check_object_ext 
that can be enabled with -Cr and -CR. So actually you do try to check 
the validity of objects.


But I see that this is an internal feature of FPC that is not to be 
executed manually and there is no assurance of correctness - it is only 
a helper feature for debugging.


With this said, I do accept that it is a no-go to check a valid 
enumeration value against its own type in the code (MyEnum is TMyEnum). 
We are left with a check of a simple value (ordinal, enumeration, 
subrange) against a different enumeration/subrange type:


(MyInt is TMyEnum)
(MyEnum2 is TMyEnum)
etc. as a shorthand for ((Ord(Low(TMyEnum)) <= MyInt) and (MyInt <= 
Ord(High(TMyEnum))).


This is still better than nothing.

+++

BUT (big BUT): I suggest to do an enumeration/subrange validity check on 
the compiler side when range checking is used. This is similar to the 
current fpc_check_object intrinsic.


When range checking is used, this check should be injected (at least) 
before every case-of and the new is/as operators if they are replaced 
with true. When range checking is disabled, the check should not be 
injected at all. Examples for range checking:


1.) case MyEnumValue of // <<< here an enumeration validity check should 
be injected
2.) IsValid := MyEnumValue is TEnumValue; //  <<< replace with TRUE but 
still insert a validity check intrinsic if range checking is used. If 
not replaced with true, do not insert validity check (so that false can 
be returned without a runtime error).


IMO such a solution is acceptable for both sides:
Your point: The clarity of the language will stay untouched because the 
new validity check will be only informative and used only for debugging 
without any assurance of correctness. (You see the difference: 
MyEnumValue is TEnumValue will raise an exception for an invalid value 
and not return false! False will be returned only for (MyInt is 
TEnumValue).)
My point: We will get an information about invalid enum values (range 
check error) instead of "undefined behavior" when debugging - and we 
will be able to find code where invalid enum values can occur and fix 
the code where these values were assigned. Although this information may 
not be (theoretically) absolutely accurate (the same as fpc_check_object 
is not absolutely accurate), it will still be pretty accurate and it 
will be a big help during the debugging.


Ondrej

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] [Suggestion] Enumeration range-check intrinsic

[Ondrej Pokorny]

On 03.07.2019 02:37, J. Gareth Moreton wrote:

On 03/07/2019 01:13, Ondrej Pokorny wrote:

On 02.07.2019 23:34, Jonas Maebe wrote:
As to your patch itself: why do you not directly compare the 
tconstexprint values directly, and use the svalue/uvalue fields 
instead?


Because I missed that they can be directly compared directly.


I dare say, I would advise you make any new changes to my 
"AS-IS-enum-04.patch", since that adds an internal error to one of the 
case blocks (mostly to suppress a compiler warning) and ensures it 
merges cleanly with the trunk.


Of course I would do it if I decided to work on it. But no, I won't work 
on the compiler bugs/extension/anything any more. It's just a waste of time.


One reason is that I don't see any reasonable philosophy behind it. As a 
result FPC is a mixture of Delphi compatibilities and incompatibilities, 
applied optimizations and missing optimizations, applied extensions and 
missing extensions that even contradict one another sometimes. I see FPC 
as a personal project of the compiler team who has an absolute right of 
taking decisions about it. So I just accept it as-is.


The second reason is that the compiler people are very busy (which I 
perfectly understand) and it's hard to get some feedback even when 
fixing bugs.


On the other hand it's not that bad all in all. FPC still is a good 
compiler and I prefer it over Delphi. And I have the source code of it, 
so if it really matters, I can patch it for myself.


Ondrej

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] [Suggestion] Enumeration range-check intrinsic

[Ondrej Pokorny]

On 02.07.2019 23:34, Jonas Maebe wrote:

Invalid data means undefined behaviour, always. "is" is not a special
case that is immune to this.


Don't you really see the need to handle invalid data with a /defined/ 
behavior? Enum and subrange types /can/ store invalid data and they /do/ 
store invalid data. Be it implicit assignments in object creation, 
clearing records with Default(TMyRecord), reading records from streams 
etc etc. The issue is here and it needs some solution. If you don't like 
"is" for this purpose, why not to introduce a compiler intrinsic 
Valid(MyValue) that would check if MyValue is within the allowed range 
of its type?


Or do you really think Valid(MyValue) should always return true and thus 
be replaced with the true constant at compile time?


Ondrej
___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] [Suggestion] Enumeration range-check intrinsic

[Ondrej Pokorny]

On 03.07.2019 08:58, Ondrej Pokorny wrote:


BTW I was always convinced that an explicit typecast switches off 
range checking. And now I open the documentation 
https://www.freepascal.org/docs-html/prog/progsu65.html and I read 
this: (citation)
/If, at run-time, an index or enumeration type is specified that is 
out of the declared range of the compiler, then a run-time error is 
generated, and the program exits with exit code 201. This can happen 
when doing a typecast (implicit or explicit) on an enumeration type or 
subrange type./



More fun:

program Project1;
{$R+}
type
  TSubRange = 1..2;
var
  S: TSubRange;
begin
  S := Default(TSubRange);
  Writeln(S);
end.

compiles and runs just fine. Why?

This does not compile:

program Project1;
{$R+}
type
  TSubRange = 1..2;
var
  S: TSubRange;
begin
  S := 0;
  Writeln(S);
end.

Ondrej

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel