Re: [fpc-devel] Managed Types, Undefined Bhaviour
On 29/06/18 22:02, Stefan Glienke wrote: If I am not mistaken (this is from my observarion and might not be 100% accurate) usually the rule in Delphi (possibly similar in FPC) when it allows to directly pass the LHS as hidden result parameter is when it is a local variable. FPC does it when it knows for certain the target cannot be read or modified by the called function. Even for local variables, this is not always the case, e.g. if the local variable's address may have been taken at some point: by the address parameter, or simply because it was passed as var- or out-parameter (the called function could then take the address of that parameter and store it elsewhere). I.e., it is based on alias analysis. With whole-program analysis, it could also be done with global variables, or in cases the compiler could prove that the function that received a (local) variable as var/out-parameter did not take its address and store it in a location that survived the lifetime of the called function. That's what I meant when I said this proposal defines language behaviour based on the limitations of a compiler's implementation of alias analysis. Nobody can be 100% accurate about this because this can change from one compiler version to another, unless you limit yourself to the very first implementation your original compiler had. Any improvement could break working code. Now with the dynamic array we have the case that since the content of the temp variable was being copied it then when being reused has a ref count of 1 causing SetLength to just realloc the memory and possibly keep any existing content of the array copying it then on to the LHS of a second statement as shown in the code example earlier. I fully agree it is counter-intuitive. But I disagree that only solving the issue in specific cases (which aren't even documented anywhere) is a solution. Either you go all the way, or you don't do it. Having the programmer keep manual track of when a managed type might need extra initialisation does not make sense. After all, when you change your code, behaviour could change because suddenly a local variable might have a value at a point where previously it didn't. Having to follow all code paths to see whether somewhere you call a function that does not initialise its function result so you can add an extra nil-initialisation of that managed local variable is the opposite of robust code. IMO there is a potential cause for a code defect that is not obvious - you might go the way of the principle of least surprises and try to address it The principle of least surprises would be to always use temps and to always initialise them before passing them as function result. I think many more people would expect this rather than "it is usually initialised to nil". It would of course result in complaints that FPC performs extra initialisations of managed variables compared to Delphi. or argue it away for whatever reasons. That is no way to discuss. Jonas ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
Re: [fpc-devel] Managed Types, Undefined Bhaviour
> -Original Message-
> From: fpc-devel On Behalf Of
> Mattias Gaertner
>
> > [...] COM interface methods can't logically not be virtual,
>
> I think you are confusing things here. They can be virtual or not
> virtual in COM and CORBA.
I think a lot of people simply don't understand how interfaces are implemented.
A decade ago, I wrote some articles about interfaces:
https://www.nexusdb.com/support/index.php?q=intf-fundamentals
https://www.nexusdb.com/support/index.php?q=intf-advanced
https://www.nexusdb.com/support/index.php?q=intf-aggregation
Looking through these now, I see that there are details missing about how
Delphi (and I assume FPC) actually implements interfaces. (The following
assumes that the read has some understanding about interfaces that can be
gained from the articles above.)
An interface being "a pointer to a pointer to an array of method pointers"
raises the question, "where is the memory that the 2nd pointer in that chain is
stored in?"
When you define a class like:
type
ISomeInterfaceA = interface(IInterface)
['{7C6DC303-0D93-4212-971F-6EACA1B97015}']
procedure SomeMethod;
end;
TSomeObjectA = class(TInterfacedObject, ISomeInterfaceA)
protected
{--- ISomeInterfaceA ---}
procedure SomeMethod;
end;
The in-memory layout of that class is something like:
VMT : Pointer { to array of method pointer};
InheritedFields : Array[x] of Byte; // fields inherited from
TInterfacedObject
TSomeObjectA_ISomeInterfaceA_VMT : Pointer { to array of method pointer};
So if you have a TSomeObjectA instance and get an ISomeInterfaceA from that,
what you get is a pointer to that TSomeObjectA_ISomeInterfaceA_VMT field of
that particular instance of TSomeObjectA.
Which raise the next question, what's the value of that
TSomeObjectA_ISomeInterfaceA_VMT and where does it come from?
The compiler, at the time it compiles TSomeObjectA will create a static array
of method pointers (for all the methods in the interface, including ancestors)
and store a pointer to that in the RTTI of the class.
When the an instance of TSomeObjectA is created, as part of the work done
before the constructor is run, the RTL will go through the RTTI of the class,
find out about all the "hidden interface VMT pointer fields" and initialize
them to point at the array of method pointers the compiler generated. So all
instances of TSomeObjectA will always have exactly the same value in the hidden
TSomeObjectA_ISomeInterfaceA_VMT field.
Which raise the next question, what code exactly are the method pointers in
that compiler generated TSomeObjectA_ISomeInterfaceA_VMT array pointing to?
A valid ISomeInterfaceA VMT is expected to contain a pointer in the SomeMethod
slot to something like:
procedure(Self: ISomeInterfaceA);
But the code for TSomeObjectA.SomeMethod is:
procedure(Self: TSomeObjectA);
So the compiler clearly can't just put a pointer to TSomeObjectA.SomeMethod
into the SomeMethod slot of the TSomeObjectA_ISomeInterfaceA_VMT. Instead, the
compiler needs to create code for a hidden trampoline that looks like this:
procedure TSomeObjectA_ISomeInterfaceA_SomeMethod(Self: ISomeInterfaceA);
begin
TSomeObjectA(PByte(Self)-Offset_of_TSomeObjectA_ISomeInterfaceA_VMT_in_instance_data).SomeMethod;
end;
So the code can reconstruct the TSomeObjectA pointer because the position of
the TSomeObjectA_ISomeInterfaceA_VMT field (which is what Self: ISomeInterfaceA
points to) is always at a fixed offset from the class VMT (which is what a
TSomeObjectA variable points to).
And now we can get to the "COM interface methods can't logically not be
virtual" statement.
A call against ISomeInterfaceA.SomeMethod is always "virtual". Yes. It always
involved looking up the appropriate method pointer in the interface VMT and
then calling that. So you can have different ISomeInterfaceA that will have
different code pointers in the SomeMethod slot of their VMT.
But that virtual call only gets you to the compiler generated trampoline. Which
then reconstructs the object pointer, and makes the actual call against the
object method. And THAT call can be static or virtual, depending if the method
in the class is virtual or not.
Let's continue with the code example from above:
type
ISomeInterfaceB = interface(ISomeInterfaceA)
['{1BB48CC2-A2AF-4C7E-A798-288B0F30F04F}']
procedure SomeOtherMethod;
end;
TSomeObjectB = class(TSomeObjectA, ISomeInterfaceB)
protected
{--- ISomeInterfaceA ---}
procedure SomeMethod; //not virtual!
{--- ISomeInterfaceB ---}
procedure SomeOtherMethod;
end;
The resulting memory layout of TSomeObjectB will be:
VMT : Pointer { to array of method pointer};
InheritedFields : Array[x] of Byte; // fields inherited from
TInterfacedObject
TSomeObjectA_ISomeInterfaceA_VMT : Pointer { to array of method pointer};
Re: [fpc-devel] Managed Types, Undefined Bhaviour
On 29/06/18 19:03, Stefan Glienke wrote: Delphi does not reuse them, every call to a function generates a temp variable. Sure, if you call it in a loop it of course uses the same one. That does not make any sense to me from a language design point of view. Either the language guarantees that managed function results are initialised to empty, or it does not. The fact that these are the same or different temps, or if there are no temps at all, should not matter in the least. Otherwise you are defining the behaviour of the language in terms of the quality of the compiler's alias analysis (since if it can prove that it does not need a temp, the function result may not be empty on entry). But if you have 2 calls after each other the compiler generates two variables. Even if they are in seperate code branches. I have often enough optimized some code that caused huge prologue/epilogue just for temp variables of different calls where only one could have happened (like in a case statement). I'm sorry, but supporting the exploitation of properties of the Delphi code generator is not in the scope of the FPC project. Jonas ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
Re: [fpc-devel] Managed Types, Undefined Bhaviour
> -Original Message- > From: fpc-devel On Behalf Of > Martok > Sent: Saturday, 30 June 2018 03:15 > To: fpc-devel@lists.freepascal.org > Subject: Re: [fpc-devel] Managed Types, Undefined Bhaviour > > Okay, then why does the calling convention change matters so much? > > Maybe a COM/CORBA thing? COM interface methods can't logically not be > virtual, and the kind of code from my example has always worked (for > me!) in FPC. > > I am confused. Which sorta ties in to the whole "surprises" thing > from before we hijacked this thread... The compiler, when building the interface VMT for a specific interface as implemented by a specific class requires that the signature of the method in the class matches the signature of the method in the interface definition. So if the _AddRef and _Release methods in IInterface/IUnknown are using cdecl and the methods in the class are stdcall, it will ignore them. (Technically the compiler shouldn't need to do that, because it has to create trampolins for all interface methods anyway to fix up the self pointer, at which point any difference in calling convention could be accounted for). What I am surprised about is that the code with mismatched calling conventions compiles at all, and decides to use TInterfacedObject._AddRef (which has the correct calling convention) when building the Interface VMT, instead of producing a compiler error saying that TChainer._AddRef has a mismatching calling convention. I would not have expected that and it feels like a bug to me. ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
Re: [fpc-devel] Managed Types, Undefined Bhaviour
On Fri, 29 Jun 2018 19:15:00 +0200 Martok wrote: > Am 29.06.2018 um 16:37 schrieb Thorsten Engler: > > The specific functions that implement an interface get baked into the class > > at the moment when the interface is defined as part of the class. This > > results in important differences in behaviour, depending if methods (in the > > class) are defined as virtual or not, and if a derived class redeclares an > > interface already declared on an ancestor or not. > Okay, then why does the calling convention change matters so much? The compiler searches interface methods in the class via the method signature, which includes the calling convention. Same as Delphi. And same as Delphi, FPC does not give a hint if there is an overload with a different calling convention. I wish there would be. > [...] COM interface methods can't logically not be virtual, I think you are confusing things here. They can be virtual or not virtual in COM and CORBA. Mattias ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
Re: [fpc-devel] Managed Types, Undefined Bhaviour
> -Original Message- > From: fpc-devel On Behalf Of > Michael Van Canneyt > Sent: Saturday, 30 June 2018 03:11 > To: FPC developers' list > Subject: Re: [fpc-devel] Managed Types, Undefined Bhaviour > > > Please explain. Exactly how does it demonstrate this ? > > What is the expected output ? > And how does current output differ from expected output ? The same code results in more calls to AddRef/Release under FPC than it does under Delphi. The executed code in FPC is still "correct", in that the reference count reaches 0 (and the object is freed) late enough. So there is no issue with "correctness". But the additional redundant calls to AddRef/Release will execute lock inc/dec or add/sub instructions. Which are very expensive. ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
Re: [fpc-devel] Managed Types, Undefined Bhaviour
On Fri, 29 Jun 2018 19:11:14 +0200 (CEST) Michael Van Canneyt wrote: > On Sat, 30 Jun 2018, Thorsten Engler wrote: > > >> -Original Message- > >> From: fpc-devel On Behalf Of > >> Michael Van Canneyt > >> Sent: Saturday, 30 June 2018 01:07 > >> To: FPC developers' list > >> Subject: Re: [fpc-devel] Managed Types, Undefined Bhaviour > >> > >> what does this demo actually demonstrate other than that the compiler > >> functions ? > > > > That there is a difference between "functions correctly" and "performs > > well"? > > Please explain. Exactly how does it demonstrate this ? Functions correctly: _AddRef is called for each interface reference and _AddRef and _Release are balanced. Performs well: need only 3 _AddRef instead of 6 Mattias ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
Re: [fpc-devel] Managed Types, Undefined Bhaviour
Am 29.06.2018 um 16:37 schrieb Thorsten Engler: > The specific functions that implement an interface get baked into the class > at the moment when the interface is defined as part of the class. This > results in important differences in behaviour, depending if methods (in the > class) are defined as virtual or not, and if a derived class redeclares an > interface already declared on an ancestor or not. Okay, then why does the calling convention change matters so much? Maybe a COM/CORBA thing? COM interface methods can't logically not be virtual, and the kind of code from my example has always worked (for me!) in FPC. I am confused. Which sorta ties in to the whole "surprises" thing from before we hijacked this thread... -- Regards, Martok ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
Re: [fpc-devel] Managed Types, Undefined Bhaviour
On Sat, 30 Jun 2018, Thorsten Engler wrote: -Original Message- From: fpc-devel On Behalf Of Michael Van Canneyt Sent: Saturday, 30 June 2018 01:07 To: FPC developers' list Subject: Re: [fpc-devel] Managed Types, Undefined Bhaviour what does this demo actually demonstrate other than that the compiler functions ? That there is a difference between "functions correctly" and "performs well"? Please explain. Exactly how does it demonstrate this ? What is the expected output ? And how does current output differ from expected output ? Michael. ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
Re: [fpc-devel] Managed Types, Undefined Bhaviour
Delphi does not reuse them, every call to a function generates a temp variable. Sure, if you call it in a loop it of course uses the same one. But if you have 2 calls after each other the compiler generates two variables. Even if they are in seperate code branches. I have often enough optimized some code that caused huge prologue/epilogue just for temp variables of different calls where only one could have happened (like in a case statement). You can see exactly that in the answer by David Heffernan you linked to. The loop keeps adding X while the other two calls get an empty string passed. Am 29.06.2018 um 18:27 schrieb Jonas Maebe: On 29/06/18 17:57, Stefan Glienke wrote: Now we are back to using temp variables (both Delphi and FPC do) but FPC again reuses its temp variable for A and B while Delphi uses different ones. Now for some integer this might not be a big issue but imagine you have something else in these arrays (for example any managed type like an interface). Not having properly cleared B because it still uses the temporary content from A might cause some issues. My point was that Delphi sometimes also reuses temp variables. See the StackOverflow posts linked from the previous message. It does not do it in the same cases as FPC, but it does do it. So while you may be lucky more often in Delphi, relying on this behaviour is unsafe even there afaik. Jonas ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel --- Diese E-Mail wurde von Avast Antivirus-Software auf Viren geprüft. https://www.avast.com/antivirus ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
Re: [fpc-devel] Managed Types, Undefined Bhaviour
On 29/06/18 17:57, Stefan Glienke wrote: Now we are back to using temp variables (both Delphi and FPC do) but FPC again reuses its temp variable for A and B while Delphi uses different ones. Now for some integer this might not be a big issue but imagine you have something else in these arrays (for example any managed type like an interface). Not having properly cleared B because it still uses the temporary content from A might cause some issues. My point was that Delphi sometimes also reuses temp variables. See the StackOverflow posts linked from the previous message. It does not do it in the same cases as FPC, but it does do it. So while you may be lucky more often in Delphi, relying on this behaviour is unsafe even there afaik. Jonas ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
Re: [fpc-devel] Bugfixes merge request
On 21.06.2018 22:34, Ondrej Pokorny wrote: Because there have been rumours about merging 3.2, I'd like to request to review 2 bug fixes that I'd like to have in 3.2 and therefore before merging: https://bugs.freepascal.org/view.php?id=33563 https://bugs.freepascal.org/view.php?id=33564 Why I request to review the issue reports: 1.) They are bug fixes targeting real issues in the Lazarus IDE component streaming. (Some properties are not correctly written to the LFM - see https://bugs.freepascal.org/view.php?id=33567) 2.) They achieve Delphi compatibility (the current state is both buggy and Delphi incompatible). 3.) I posted patches so you only need to review them. 4.) I posted test case projects as well. Nobody? Ondrej ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
Re: [fpc-devel] Managed Types, Undefined Bhaviour
Let me add some information to this issue - as I think this is one - before this drifted into the interface chaining thing: When you execute this code in Delphi it will print 0 while on FPC it prints 42: type Vector = array of integer; function DoSomething (len: Integer): Vector; begin SetLength(Result, len); end; var A, B: Vector; begin A := DoSomething(3); A[0] := 42; B := DoSomething(4); Writeln(B[0]); end. This is the case as FPC reuses the same temp variable for both calls and thus still points to an allocated array from the first call which it just enlarges by one. Delphi in this case does not directly pass A and B but does temp variables for both (it does that because A and B are global variables). When you move this code to a routine like this: procedure Main; var A, B: Vector; begin A := DoSomething(3); A[0] := 42; B := DoSomething(4); Writeln(B[0]); end; begin Main; end. both Delphi and FPC behave the same - both directly pass A and B do DoSomething because they are local variables that can not be affected by any other code running at the same time. While this might look as something that does not have any serious implication it actually might have. Change the code to following: type TFoo = class A, B: Vector; end; procedure Main; var foo: TFoo; begin foo := TFoo.Create; foo.A := DoSomething(3); foo.A[0] := 42; foo.B := DoSomething(4); Writeln(foo.B[0]); end; Now we are back to using temp variables (both Delphi and FPC do) but FPC again reuses its temp variable for A and B while Delphi uses different ones. Now for some integer this might not be a big issue but imagine you have something else in these arrays (for example any managed type like an interface). Not having properly cleared B because it still uses the temporary content from A might cause some issues. > On 28 June 2018 at 20:06 Jonas Maebe wrote: > > > On 28/06/18 18:45, Willibald Krenn wrote: > > > > type Vector = array of integer; > > function DoSomething (len: longint): Vector; begin > >SetLength (result,len); // whatever > > end; > > > > var A,B: Vector; > > begin > >A := DoSomething(3); > >// changing A here will influence B below, > >// which seems ok to some as > >// "managed types are not initialized when > >// calling DoSomething and this is documented > >// - so anything goes and stop whining". > >B := DoSomething(3); > > end. > > > > I strongly believe that the behaviour as currently implemented is wrong > > and here is why. > > (1) When assigning "result" to A, refcount of result needs to go down > > and, hence, the local var needs to be freed/invalidated. (Result goes > > out of scope.) > > For performance reasons, managed function results are implemented as > call-by-reference parameters. This is also the case in Delphi: > http://docwiki.embarcadero.com/RADStudio/Berlin/en/Program_Control#Handling_Function_Results > > Therefore the function result's scope is the caller of the function, not > the function itself. > > > (3) The behaviour is incompatible to Delphi (tested 10.2.3 vs. Lazarus > > 1.8.0). > > It is compatible with Delphi. See e.g. > https://www.mail-archive.com/fpc-pascal@lists.freepascal.org/msg43050.html > > > Delphi shows the expected behaviour of treating A and B as distinct. > > That's because your example gets optimized better by Delphi than by FPC: > Delphi directly passes A resp. B as the function result "var" parameter > to the function calls in your example, while FPC passes a/the same > temporary variable to them and then assigns this temprary variable to A/B. > > If you have more complex code where the Delphi compiler cannot be > certain that the variable to which you assign the function result isn't > used inside the function as well, it will behave in a similar way (as > discussed in the StackOverflow posts linked from the message above). > > > Jonas > ___ > fpc-devel maillist - fpc-devel@lists.freepascal.org > http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
Re: [fpc-devel] Managed Types, Undefined Bhaviour
> -Original Message- > From: fpc-devel On Behalf Of > Michael Van Canneyt > Sent: Saturday, 30 June 2018 01:07 > To: FPC developers' list > Subject: Re: [fpc-devel] Managed Types, Undefined Bhaviour > > what does this demo actually demonstrate other than that the compiler > functions ? That there is a difference between "functions correctly" and "performs well"? ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
Re: [fpc-devel] Managed Types, Undefined Bhaviour
On Fri, 29 Jun 2018, Mattias Gaertner wrote:
Pressed send too quickly.
home:~> ./tirc
Chain: 7FA5948CF040
Chain: 7FA5948CF040
Done: 7FA5948CF040
"stdcall" is wrong for Linux. It must be
{$IFNDEF WINDOWS}cdecl{$ELSE}stdcall{$ENDIF};
Then you get under Linux:
Addref: 7F0B935BF040 Refcount: 1 at 0041331A
Addref: 7F0B935BF040 Refcount: 2 at 004121F2
Release: 7F0B935BF040 Refcount: 2 at 00412196
Chain: 7F0B935BF040
Addref: 7F0B935BF040 Refcount: 2 at 0041331A
Addref: 7F0B935BF040 Refcount: 3 at 004121F2
Release: 7F0B935BF040 Refcount: 3 at 00412196
Chain: 7F0B935BF040
Addref: 7F0B935BF040 Refcount: 3 at 0041331A
Addref: 7F0B935BF040 Refcount: 4 at 004121F2
Release: 7F0B935BF040 Refcount: 4 at 00412196
Done: 7F0B935BF040
fin
Release: 7F0B935BF040 Refcount: 3 at 00412196
Release: 7F0B935BF040 Refcount: 2 at 00412196
Release: 7F0B935BF040 Refcount: 1 at 00412196
ahahahahahaha... Good point. Changed it and now I get this too.
Explains a lot. Strange the compiler does not warn about this.
("hides method of parent class" or somesuch)
With that out of the way, the original question still remains:
what does this demo actually demonstrate other than that the compiler functions
?
Michael.
___
fpc-devel maillist - fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
Re: [fpc-devel] Managed Types, Undefined Bhaviour
On Fri, 29 Jun 2018 16:18:04 +0200 (CEST)
Michael Van Canneyt wrote:
> On Fri, 29 Jun 2018, Michael Van Canneyt wrote:
>
> >
> >
> > On Fri, 29 Jun 2018, Martok wrote:
> >
> >> Am 29.06.2018 um 14:41 schrieb Michael Van Canneyt:
> >>> As far as I can see, you get 2 chain and 1 done call. Which is what I'd
> > expect.
> >>> The overrides of the _* calls are useless, since they are not virtual in
> >>> TInterfacedObject and hence never called. So that's OK too.
> >> Interface functions are always virtual and implemented by the actually
> >> instantiated class.
>
> > The "override" keyword is neither allowed nor needed, this
> >> code does what it looks like.
> >>
> >> The expected output would be 3 Addrefs and 3 Releases.
> >
> > I don't get that.
>
> Pressed send too quickly.
>
> home:~> ./tirc
> Chain: 7FA5948CF040
> Chain: 7FA5948CF040
> Done: 7FA5948CF040
"stdcall" is wrong for Linux. It must be
{$IFNDEF WINDOWS}cdecl{$ELSE}stdcall{$ENDIF};
Then you get under Linux:
Addref: 7F0B935BF040 Refcount: 1 at 0041331A
Addref: 7F0B935BF040 Refcount: 2 at 004121F2
Release: 7F0B935BF040 Refcount: 2 at 00412196
Chain: 7F0B935BF040
Addref: 7F0B935BF040 Refcount: 2 at 0041331A
Addref: 7F0B935BF040 Refcount: 3 at 004121F2
Release: 7F0B935BF040 Refcount: 3 at 00412196
Chain: 7F0B935BF040
Addref: 7F0B935BF040 Refcount: 3 at 0041331A
Addref: 7F0B935BF040 Refcount: 4 at 004121F2
Release: 7F0B935BF040 Refcount: 4 at 00412196
Done: 7F0B935BF040
fin
Release: 7F0B935BF040 Refcount: 3 at 00412196
Release: 7F0B935BF040 Refcount: 2 at 00412196
Release: 7F0B935BF040 Refcount: 1 at 00412196
Mattias
___
fpc-devel maillist - fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
Re: [fpc-devel] Managed Types, Undefined Bhaviour
> -Original Message-
> From: fpc-devel On Behalf Of
> Martok
> Sent: Friday, 29 June 2018 23:55
> To: fpc-devel@lists.freepascal.org
> Interface functions are always virtual and implemented by the
> actually instantiated class. The "override" keyword is neither
> allowed nor needed,
Without having looked the particular code this thread is about, that statement,
at least how I interpret it, is wrong.
The specific functions that implement an interface get baked into the class at
the moment when the interface is defined as part of the class. This results in
important differences in behaviour, depending if methods (in the class) are
defined as virtual or not, and if a derived class redeclares an interface
already declared on an ancestor or not.
I've only tried the following code (which demonstrates this) in Delphi, but
would assume FPC to produce the same result (otherwise there is bound to be a
lot of Delphi code which produces subtly different outcomes when compiled with
FPC).
program IntfImplDetails;
{$APPTYPE CONSOLE}
uses
System.SysUtils;
type
IFoo = interface(IInterface)
['{E9A12596-8F61-4CF1-A09A-266D56BD837D}']
procedure Foo;
end;
IBar = interface(IFoo)
['{6782527D-431E-49F4-89D0-DCF871BE63A3}']
procedure Bar;
end;
TFoo = class(TInterfacedObject, IFoo)
protected
procedure Foo;
end;
TFooBar = class(TFoo, IBar)
protected
procedure Bar;
procedure Foo;
end;
TFooBarToo = class(TFooBar, IFoo)
protected
procedure Bar;
procedure Foo;
end;
TVirtFoo = class(TInterfacedObject, IFoo)
protected
procedure Foo; virtual;
end;
TVirtFooBar = class(TVirtFoo, IBar)
protected
procedure Bar;
procedure Foo; override;
end;
{ TFoo }
procedure TFoo.Foo;
begin
WriteLn('TFoo.Foo');
end;
procedure TFooBar.Bar;
begin
WriteLn('TFooBar.Bar');
end;
procedure TFooBar.Foo;
begin
WriteLn('TFooBar.Foo');
end;
procedure TFooBarToo.Bar;
begin
WriteLn('TFooBarToo.Bar');
end;
procedure TFooBarToo.Foo;
begin
WriteLn('TFooBarToo.Foo');
end;
procedure TVirtFoo.Foo;
begin
WriteLn('TVirtFoo.Foo');
end;
procedure TVirtFooBar.Bar;
begin
WriteLn('TVirtFooBar.Bar');
end;
procedure TVirtFooBar.Foo;
begin
WriteLn('TVirtFooBar.Foo');
end;
var
Intf : IInterface;
IntfFoo : IFoo;
IntfBar : IBar;
begin
try
{=== TFooBar ===}
WriteLn('=== TFooBar ===');
Intf := TFooBar.Create;
Supports(Intf, IFoo, IntfFoo);
IntfFoo.Foo; // TFoo.Foo
Supports(Intf, IBar, IntfBar);
IntfBar.Foo; // TFooBar.Foo
IntfBar.Bar; // TFooBar.Bar
IntfFoo := IntfBar;
IntfFoo.Foo; // TFooBar.Foo
{=== TFooBarToo ===}
WriteLn('=== TFooBarToo ===');
Intf := TFooBarToo.Create;
Supports(Intf, IFoo, IntfFoo);
IntfFoo.Foo; // TFooBarToo.Foo
Supports(Intf, IBar, IntfBar);
IntfBar.Foo; // TFooBar.Foo
IntfBar.Bar; // TFooBar.Bar
IntfFoo := IntfBar;
IntfFoo.Foo; // TFooBar.Foo
{=== TVirtFooBar ===}
WriteLn('=== TVirtFooBar ===');
Intf := TVirtFooBar.Create;
Supports(Intf, IFoo, IntfFoo);
IntfFoo.Foo; // TVirtFooBar.Foo
Supports(Intf, IBar, IntfBar);
IntfBar.Foo; // TVirtFooBar.Foo
IntfBar.Bar; // TVirtFooBar.Bar
IntfFoo := IntfBar;
IntfFoo.Foo; // TVirtFooBar.Foo
except
on E: Exception do
Writeln(E.ClassName, ': ', E.Message);
end;
if DebugHook <> 0 then
ReadLn;
end.
___
fpc-devel maillist - fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
Re: [fpc-devel] Managed Types, Undefined Bhaviour
On Fri, 29 Jun 2018, Michael Van Canneyt wrote: On Fri, 29 Jun 2018, Michael Van Canneyt wrote: On Fri, 29 Jun 2018, Martok wrote: Am 29.06.2018 um 14:41 schrieb Michael Van Canneyt: As far as I can see, you get 2 chain and 1 done call. Which is what I'd expect. The overrides of the _* calls are useless, since they are not virtual in TInterfacedObject and hence never called. So that's OK too. Interface functions are always virtual and implemented by the actually instantiated class. The "override" keyword is neither allowed nor needed, this code does what it looks like. The expected output would be 3 Addrefs and 3 Releases. I don't get that. Pressed send too quickly. home:~> ./tirc Chain: 7FA5948CF040 Chain: 7FA5948CF040 Done: 7FA5948CF040 fin is the complete output. So either your explanation is wrong, or the compiler completely faulty. Compiling with memleak detection: home:~> fpc -glh tirc.pp home:~> ./tirc Chain: 7F6FA90280C0 Chain: 7F6FA90280C0 Done: 7F6FA90280C0 fin Heap dump by heaptrc unit of /home/michael/tirc 1 memory blocks allocated : 32/32 1 memory blocks freed : 32/32 0 unfreed memory blocks : 0 True heap size : 32768 True free heap : 32768 Tested on Delphi. Seems FPC is completely faulty. Delphi calls the versions in TChainer, FPC does not. Well, You can guess how often this feature is used then... :) Michael. ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
Re: [fpc-devel] Managed Types, Undefined Bhaviour
Am 29.06.2018 um 16:05 schrieb Michael Van Canneyt: >> The expected output would be 3 Addrefs and 3 Releases. > > I don't get that. Somewhat current FPC trunk output, annotations added manually: == Addref: 0022FAA8 Refcount: 1 at 00404961 (by fpc_class_as_intf in GetChainer) Addref: 0022FAA8 Refcount: 2 at 00404223 (by fpc_intf_assign of GetChainer Result) Release: 0022FAA8 Refcount: 2 at 004041F4 (by fpc_intf_decr_ref of GetChainer Result) Chain: 0022FAA8 Addref: 0022FAA8 Refcount: 2 at 00404961 (by fpc_class_as_intf in Chain) Addref: 0022FAA8 Refcount: 3 at 00404223 (by fpc_intf_assign of Chain Result) Release: 0022FAA8 Refcount: 3 at 004041F4 (by fpc_intf_decr_ref of Chain Result) Chain: 0022FAA8 Addref: 0022FAA8 Refcount: 3 at 00404961 Addref: 0022FAA8 Refcount: 4 at 00404223 Release: 0022FAA8 Refcount: 4 at 004041F4 Done: 0022FAA8 fin Release: 0022FAA8 Refcount: 3 at 004041F4 (by fpc_intf_decr_ref at scope end of Test) Release: 0022FAA8 Refcount: 2 at 004041F4 (dito) Release: 0022FAA8 Refcount: 1 at 004041F4 (dito) == Delphi output (without the stack trace part, because they don't have it): == Addref: 0205DBE8 Refcount: 1 Chain: 0205DBE8 Addref: 0205DBE8 Refcount: 2 Chain: 0205DBE8 Addref: 0205DBE8 Refcount: 3 Done: 0205DBE8 fin Release: 0205DBE8 Refcount: 3 Release: 0205DBE8 Refcount: 2 Release: 0205DBE8 Refcount: 1 == Delphi uses a shortcut for "as", and because of their different handling of putting Result in a tempvar, they get away with less internal assignments. 6 LOCK instructions (and associated calls) less. Not a lot by itself, but since we're counting single-digit cycles in other places... -- Regards, Martok ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
Re: [fpc-devel] Managed Types, Undefined Bhaviour
On Fri, 29 Jun 2018, Michael Van Canneyt wrote: On Fri, 29 Jun 2018, Martok wrote: Am 29.06.2018 um 14:41 schrieb Michael Van Canneyt: As far as I can see, you get 2 chain and 1 done call. Which is what I'd expect. The overrides of the _* calls are useless, since they are not virtual in TInterfacedObject and hence never called. So that's OK too. Interface functions are always virtual and implemented by the actually instantiated class. The "override" keyword is neither allowed nor needed, this code does what it looks like. The expected output would be 3 Addrefs and 3 Releases. I don't get that. Pressed send too quickly. home:~> ./tirc Chain: 7FA5948CF040 Chain: 7FA5948CF040 Done: 7FA5948CF040 fin is the complete output. So either your explanation is wrong, or the compiler completely faulty. Compiling with memleak detection: home:~> fpc -glh tirc.pp home:~> ./tirc Chain: 7F6FA90280C0 Chain: 7F6FA90280C0 Done: 7F6FA90280C0 fin Heap dump by heaptrc unit of /home/michael/tirc 1 memory blocks allocated : 32/32 1 memory blocks freed : 32/32 0 unfreed memory blocks : 0 True heap size : 32768 True free heap : 32768 Michael. ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
Re: [fpc-devel] Managed Types, Undefined Bhaviour
On Fri, 29 Jun 2018, Martok wrote: Am 29.06.2018 um 14:41 schrieb Michael Van Canneyt: As far as I can see, you get 2 chain and 1 done call. Which is what I'd expect. The overrides of the _* calls are useless, since they are not virtual in TInterfacedObject and hence never called. So that's OK too. Interface functions are always virtual and implemented by the actually instantiated class. The "override" keyword is neither allowed nor needed, this code does what it looks like. The expected output would be 3 Addrefs and 3 Releases. I don't get that. Michael. ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
Re: [fpc-devel] Managed Types, Undefined Bhaviour
Am 29.06.2018 um 14:41 schrieb Michael Van Canneyt: > As far as I can see, you get 2 chain and 1 done call. Which is what I'd > expect. > The overrides of the _* calls are useless, since they are not virtual in > TInterfacedObject and hence never called. So that's OK too. Interface functions are always virtual and implemented by the actually instantiated class. The "override" keyword is neither allowed nor needed, this code does what it looks like. The expected output would be 3 Addrefs and 3 Releases. A bit better would be doing the last release after "Done" and before "fin" (but this is difficult because of the implied exception frame - there are cases where Delphi does it anyway, depending on method length). The "ideal" output would get away with even less (but I don't think this is possible without translating to SSA form first and doing some strict counting). The observed output is 6 Addrefs and 6 Releases. At some point in the past (this may have to do with variable and register allocation), a Release could happen too soon. It doesn't now, so that's good. There is nothing technically wrong with the generated code, but it is not nearly as efficient as it could be. See also Ryan's comments about slow Interlocked*-calls a few weeks ago. Delphi's output for the same example is better, giving the expected output. Because of the tempvars, it is also not exactly what one might expect at first, which is why I mentioned it in this context. -- Regards, Martok ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
Re: [fpc-devel] Managed Types, Undefined Bhaviour
On Fri, 29 Jun 2018, Martok wrote: Am 29.06.2018 um 12:43 schrieb Michael Van Canneyt: Out of curiosity, can you give a simple example of such a funny behaviour in such a chaining pattern ? We've had this topic about 2 years ago with regard to automatic file close on interface release. Interestingly, something must have changed in the mean time, because the trivial testcase is now *different* , which is somewhat the point of being weird-undefined ;-) Take this example: https://pastebin.com/gsdVXWAi The tempvar used to get reused, causing lifetime issues with the "chain object". This isn't the case anymore, now three independent tempvars are used, all of which live until the end of the function, potentially keeping the object alive for a long time. There is also one fpc_intf_assign with associated addref/release per as operator, which isn't technically necessary. One could probably avoid the interfaces here with ARC records, but either I'm missing something or the scope lifetime of tempvars there is even worse. What is the expected output of this program ? As far as I can see, you get 2 chain and 1 done call. Which is what I'd expect. The overrides of the _* calls are useless, since they are not virtual in TInterfacedObject and hence never called. So that's OK too. There is no memory leak, output is the same with 2.6.4 and 3.0.4 and trunk, so what does this demo actually demonstrate other than that the code just works ? Michael. ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
Re: [fpc-devel] Managed Types, Undefined Bhaviour
Am 29.06.2018 um 12:43 schrieb Michael Van Canneyt: > Out of curiosity, can you give a simple example of such a funny behaviour > in such a chaining pattern ? We've had this topic about 2 years ago with regard to automatic file close on interface release. Interestingly, something must have changed in the mean time, because the trivial testcase is now *different* , which is somewhat the point of being weird-undefined ;-) Take this example: https://pastebin.com/gsdVXWAi The tempvar used to get reused, causing lifetime issues with the "chain object". This isn't the case anymore, now three independent tempvars are used, all of which live until the end of the function, potentially keeping the object alive for a long time. There is also one fpc_intf_assign with associated addref/release per as operator, which isn't technically necessary. One could probably avoid the interfaces here with ARC records, but either I'm missing something or the scope lifetime of tempvars there is even worse. -- Regards, Martok ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
Re: [fpc-devel] Possible internal corruption
So I've made a breakthrough. The memory corruption is due to both parts of the CMOV optimization under OptPass2Jcc, not my Jcc addition (although it might have unintentionally accentuated it). The first part sets p to a dangling pointer, while the 2nd part is a little more complicated, but I'll try to spell everything out once I finish testing my new patch and see if I've eliminated all of my crashes Gareth On Fri 29/06/18 10:05 , "J. Gareth Moreton" gar...@moreton-family.com sent: It turns out that it's invalid memory. Trying to call "ClassName" raises an access violation (other aligns work fine). There's a dangling pointer somewhere. I found one in the CMOV optimisation code, but that hasn't fixed the crash. Gareth On Fri 29/06/18 10:27 , Martok list...@martoks-place.de sent: > A clue that leads me to believe there's internal corruption is that a produced > .s file yields an alignment field of ".balign 119,0x90", which should never > happen. Could you set a breakpoint on aggas.pas:721 (the call to doalign) with a conditional on "tai_align_abstract(hp).aligntype=119" and check what the actual type of hp is? It could be that at some point a node gets its typ changed in an invalid way? aligntype should be either one of 2^[0..5], never something else... This is where AddressSanitizer support would be *nice*. -- Regards, Martok Ceterum censeo b32079 esse sanandam. ___ fpc-devel maillist - fpc-devel@lists.freepascal.org [1] http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel [2]">http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel ___ fpc-devel maillist - fpc-devel@lists.freepascal.org [3] http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel [4]">http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel Links: -- [1] mailto:fpc-devel@lists.freepascal.org [2] http://secureweb.fast.net.uk/ http:= [3] mailto:fpc-devel@lists.freepascal.org [4] http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
Re: [fpc-devel] Managed Types, Undefined Bhaviour
On Fri, 29 Jun 2018, Martok wrote: I hope this issue gets addressed, as I deem the current behaviour completely broken and also going totally against the spirit of Pascal, feeling much more like some very obscure behaviour I'd expect from some C compiler. Discovering the handling of this issue, however, makes me wonder whether fpc aims to be a great Pascal compiler that does without bad surprises and very very hard to debug "documented" behaviour or not. There is less undefined behaviour than in C, but the one we have will bite you in the most awful ways, sometimes after years of working just fine. And we don't even have a nice formal standards document that one could grep for "undefined". But yeah, as Jonas wrote, this _isn't_ one of these occasions. FPC uses (and reuses!) tempvars a lot more than Delphi, which causes all sorts of funny behaviours with managed types. Try returning a string or use the JavaScript-style "Foo().Bar().Baz()" method chaining pattern and you'll see what I mean. Out of curiosity, can you give a simple example of such a funny behaviour in such a chaining pattern ? Michael.___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
Re: [fpc-devel] Possible internal corruption
It turns out that it's invalid memory. Trying to call "ClassName" raises an access violation (other aligns work fine). There's a dangling pointer somewhere. I found one in the CMOV optimisation code, but that hasn't fixed the crash. Gareth On Fri 29/06/18 10:27 , Martok list...@martoks-place.de sent: > A clue that leads me to believe there's internal corruption is that a produced > .s file yields an alignment field of ".balign 119,0x90", which should never > happen. Could you set a breakpoint on aggas.pas:721 (the call to doalign) with a conditional on "tai_align_abstract(hp).aligntype=119" and check what the actual type of hp is? It could be that at some point a node gets its typ changed in an invalid way? aligntype should be either one of 2^[0..5], never something else... This is where AddressSanitizer support would be *nice*. -- Regards, Martok Ceterum censeo b32079 esse sanandam. ___ fpc-devel maillist - fpc-devel@lists.freepascal.org [1] http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel [2]">http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel Links: -- [1] mailto:fpc-devel@lists.freepascal.org [2] http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
Re: [fpc-devel] Managed Types, Undefined Bhaviour
> I hope this issue gets addressed, as I deem the current behaviour completely > broken and also going totally against the spirit of Pascal, feeling much more > like some very obscure behaviour I'd expect from some C compiler. > Discovering the handling of this issue, however, makes me wonder > whether fpc aims to be a great Pascal compiler that does without bad surprises > and very very hard to debug "documented" behaviour or not. There is less undefined behaviour than in C, but the one we have will bite you in the most awful ways, sometimes after years of working just fine. And we don't even have a nice formal standards document that one could grep for "undefined". But yeah, as Jonas wrote, this _isn't_ one of these occasions. FPC uses (and reuses!) tempvars a lot more than Delphi, which causes all sorts of funny behaviours with managed types. Try returning a string or use the JavaScript-style "Foo().Bar().Baz()" method chaining pattern and you'll see what I mean. Change your function to the following, and it will do what one would expect: function DoSomething (len: longint): Vector; begin Result:= nil; SetLength (result,len); // whatever end; For managed types, as far as I can tell: - locals are initialized (even if there is a warning telling you they are not) - tempvars are initialized *once* - Result is never initialized (there is no warning telling you it is not). -- Regards, Martok Ceterum censeo b32079 esse sanandam. ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
Re: [fpc-devel] Possible internal corruption
> A clue that leads me to believe there's internal corruption is that a produced > .s file yields an alignment field of ".balign 119,0x90", which should never > happen. Could you set a breakpoint on aggas.pas:721 (the call to doalign) with a conditional on "tai_align_abstract(hp).aligntype=119" and check what the actual type of hp is? It could be that at some point a node gets its typ changed in an invalid way? aligntype should be either one of 2^[0..5], never something else... This is where AddressSanitizer support would be *nice*. -- Regards, Martok Ceterum censeo b32079 esse sanandam. ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
