Re: [fpc-pascal] reference-counted function results not initialized to nil

[David Emerson]

Philosophical arguments aside, I filed a bug against the documentation, 
since one thing is clear-- that I and others were previously misled into 
believing something that is not true.


http://mantis.freepascal.org/view.php?id=30321

In the report, I included several references where the documentation 
could be improved to make the compiler's behavior more clear. If anyone 
wants to expand on that list, please feel free :)


Also perhaps the wiki could use some work in this regard.

~David.


___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

Re: [fpc-pascal] reference-counted function results not initialized to nil

[Sven Barth]

On 26.06.2016 18:20, Jürgen Hestermann wrote:
> Am 2016-06-26 um 14:13 schrieb Sven Barth:
>> But also no word that it is true either, cause you're not declaring a
> variable for the Result.
> 
> Of course I am declaring a variable:
> 
> function X : String;
> 
> Now the (local) variable "Result" is of type "String".
> I use it in the same way as any other local variable.
> And because it is a manged type I could rely on the documentation
> which says that managed types are initialized (with zero length/nil).
> "Result" is used in the same way as a (local) variable.
> What should make me think that it is different?

Variables are declared with "var", a function result is not declared
with "var".


Regards,
Sven

___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

Re: [fpc-pascal] reference-counted function results not initialized to nil

[Karoly Balogh (Charlie/SGR)]

Hi,

On Sun, 26 Jun 2016, Jürgen Hestermann wrote:

> Am 2016-06-26 um 14:27 schrieb Karoly Balogh (Charlie/SGR):
> > There's no exception. At least not specifically for Result. It is simply
> > variable passed by reference from the caller side, therefore it's not
> > initialized inside the function.
>
> You mean when I have the following:
>
> --
> function X : ansistring;
> begin
> end;
>
> var S : ansistring;
>
> S := X;
> --
>
> Then S is a dangling pointer to nowhere?

No. In this case, S was initialized (to nil), because it's a global var.
This just won't be changed by the X function call. If you had S:='test';
Before S:=X; then it would still contain 'test' after it.

Charlie___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

Re: [fpc-pascal] reference-counted function results not initialized to nil

[Dimitrios Chr. Ioannidis]

Hi,

On 26/6/2016 7:20 μμ, Jürgen Hestermann wrote:

And because it is a manged type I could rely on the documentation
which says that managed types are initialized (with zero length/nil).
"Result" is used in the same way as a (local) variable.
What should make me think that it is different?


as Barry Kelly pointed out there 
http://stackoverflow.com/questions/861045/which-variables-are-initialized-when-in-delphi/861178#861178,
initialization differs on what mechanism was used to allocate memory . 
AFAIU, "initialized" <> "zero-filled" .


regards,

--
Dimitrios Chr. Ioannidis
www.nephelae.eu
___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

Re: [fpc-pascal] reference-counted function results not initialized to nil

[Jürgen Hestermann]

Am 2016-06-26 um 14:27 schrieb Karoly Balogh (Charlie/SGR):
> There's no exception. At least not specifically for Result. It is simply
> variable passed by reference from the caller side, therefore it's not
> initialized inside the function.

You mean when I have the following:

--
function X : ansistring;
begin
end;

var S : ansistring;

S := X;
--

Then S is a dangling pointer to nowhere?


> And this is A., consistent with other var
> parameters B., apparently also how Delphi does it. Actually, since managed
> types are always passed by reference, this is not really a surprise, nor
> an exception.

Of course it is a surprise because the documenation says that managed types are 
always initialized.
If I use a var parameter I am forced to use a variable (declared somewhere 
else) but
the function result can also be used in expressions without any (direct) 
assignment to a variable.
So it must behave like a locally declared variable IMO.

If I use an intermediate variable like:

--
function X : ansistring;
var X1 : ansistring;
begin
Result := X1+'A';
end;
--

then it is initialized while here

--
function X : ansistring;
begin
Result := Result+'A';
end;
--

I get garbage?
That's a very strange behaviour and no longer Pascal, it's C with all its funny 
side effects.

> var
>   a: ansistring;
> a:=some_function(a);

When you have a parameter then of course it depends
on how the parameter is declared.

The Result variable within a function is different
and not a parameter. You cannot feed in any data here.
I would consider this more like an out parameter
but in case of managed types an initialization
must take place somewhere (IMO in the function like
for all other local variables too, where else?).

And it is definitly a change with FPC 3 because all my programs relied
on that managed types are initialized and suddenly they are not anymore.
I am wondering what other surprises lurk here and there which I have just not 
found out.

___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

Re: [fpc-pascal] reference-counted function results not initialized to nil

[Jürgen Hestermann]

Am 2016-06-26 um 14:13 schrieb Sven Barth:
> But also no word that it is true either, cause you're not declaring a 
variable for the Result.

Of course I am declaring a variable:

function X : String;

Now the (local) variable "Result" is of type "String".
I use it in the same way as any other local variable.
And because it is a manged type I could rely on the documentation
which says that managed types are initialized (with zero length/nil).
"Result" is used in the same way as a (local) variable.
What should make me think that it is different?

___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

Re: [fpc-pascal] reference-counted function results not initialized to nil

[Martin Schreiber]

On Sunday 26 June 2016 12:09:58 Jc3bcrgen Hestermann wrote:
>
> The only exception (at least since FPC 3) is the function result
> which is totally unexpected.
> Why such an exception?
>
I don't think one could treat a function result as a normal local variable. 
The reason why not to initialise the result variable is because in many cases 
it is not necessary and reduces performance.

Martin
___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

Re: [fpc-pascal] reference-counted function results not initialized to nil

[Karoly Balogh (Charlie/SGR)]

Hi,

On Sun, 26 Jun 2016, Jürgen Hestermann wrote:

Ok, scrap what I wrote before... :|

You are right about the managed types initialization. But then the
documentation needs to be corrected there too. But actually, result is not
a local var, but in fact a "special" parameter of the function. My bad.
See below.

> The only exception (at least since FPC 3) is the function result
> which is totally unexpected.
> Why such an exception?

There's no exception. At least not specifically for Result. It is simply
variable passed by reference from the caller side, therefore it's not
initialized inside the function. And this is A., consistent with other var
parameters B., apparently also how Delphi does it. Actually, since managed
types are always passed by reference, this is not really a surprise, nor
an exception.

Actually, I tend to agree that it is very misleading. Also, you are also
right that this is new behavior in current compiler. At least 2.6.2 (I
don't have 2.6.4 at hand) decreased the reference to the result parameter
before handing it to the called function, which no longer seems to happen.
(I'm looking at the generated code.)

I have vague memories of a lengthy thread related to this, which changed
the refcounting behavior of reference passed managed types/strings, maybe
we're observing a sideeffect of this? (I think it was related to what
happens if the same reference counted variable is passed to multiple var
types and/or results, maybe also out parameters were involved. It's a
pretty grey area, which is not really documented anywhere...)

To show the problem, what happens with:

var
  a: ansistring;
a:=some_function(a);

If the caller code destroys the result variable (by decreasing the
reference count, because it's a result, so lets initialize it) it won't be
valid by the time you pass it as parameter "a', because only the function
itself will increase its reference then... Also, depending on the order of
parameter passing, you might get different results on different CPU
platforms... (Fix me?) Of course the function itself could also initialize
the result, but we're back to square one with that, if result and one of
the parameters are the same, weird things might happen.

I wish someone with real clue of calling conventions would comment. Jonas
maybe?

Charlie___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

Re: [fpc-pascal] reference-counted function results not initialized to nil

[Sven Barth]

Am 26.06.2016 12:10 schrieb "Jürgen Hestermann" :
>
> Am 2016-06-26 um 01:19 schrieb Karoly Balogh (Charlie/SGR):
> > No. Local variables are not initialized by default. Managed type or not,
> > doesn't make a difference.
>
> This is not true.
> Local variables of managed types are of course initialized!
> If you declare
>
> var S : String;
>
> then it is of course initialized to nil.
> Otherwise the (random) pointer stored in S would be used
> to change the reference counter when you assign:
>
> S := '';
>
> The only exception (at least since FPC 3) is the function result
> which is totally unexpected.
> Why such an exception?
>
> And it is not even documented (at least not within FPC)!
> Which I find a bit strange because it is very important to know.
> Omitting such information in the documentation is not good.
> Here
> http://www.freepascal.org/docs-html/ref/refsu15.html
> it says:
>
> "When declaring a variable of a dynamic array type, the initial length of
the array is zero."
>
> but no word about that this is not true when this array is a the result
of a function.

But also no word that it is true either, cause you're not declaring a
variable for the Result.

Regards,
Sven
___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

Re: [fpc-pascal] reference-counted function results not initialized to nil

[C Western]

On 26/06/16 11:09, Jürgen Hestermann wrote:

Am 2016-06-26 um 01:19 schrieb Karoly Balogh (Charlie/SGR):

No. Local variables are not initialized by default. Managed type or not,
doesn't make a difference.


This is not true.
Local variables of managed types are of course initialized!
If you declare

var S : String;

then it is of course initialized to nil.
Otherwise the (random) pointer stored in S would be used
to change the reference counter when you assign:

S := '';


My understanding (having been bitten by this) is that the requirement is 
only that S has a "sensible" value, so that the heap is not corrupted on 
the assignment, as you indicate. The compiler can re-use another 
variable instead, so the unset value is some other valid string.


Colin
___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

Re: [fpc-pascal] reference-counted function results not initialized to nil

[Jürgen Hestermann]

Am 2016-06-26 um 01:19 schrieb Karoly Balogh (Charlie/SGR):
> No. Local variables are not initialized by default. Managed type or not,
> doesn't make a difference.

This is not true.
Local variables of managed types are of course initialized!
If you declare

var S : String;

then it is of course initialized to nil.
Otherwise the (random) pointer stored in S would be used
to change the reference counter when you assign:

S := '';

The only exception (at least since FPC 3) is the function result
which is totally unexpected.
Why such an exception?

And it is not even documented (at least not within FPC)!
Which I find a bit strange because it is very important to know.
Omitting such information in the documentation is not good.
Here
http://www.freepascal.org/docs-html/ref/refsu15.html
it says:

"When declaring a variable of a dynamic array type, the initial length of the array 
is zero."

but no word about that this is not true when this array is a the result of a 
function.

___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

Re: [fpc-pascal] reference-counted function results not initialized to nil

[Karoly Balogh (Charlie/SGR)]

Hi,

On Sat, 25 Jun 2016, Jürgen Hestermann wrote:

> This has definitely changed with Free Pascal 3 as my programs suddenly
> did not work anymore and I had to add Setlength() all over my code.
> Before we could rely on that managed types where always initialized
> (well, that's the purpose of managed types, no?).

No. Local variables are not initialized by default. Managed type or not,
doesn't make a difference. Result is just another local variable in this
case. I think the compiler will even warn about using uninitialized local
variables, at least with -O3 and above (or when Data Flow Analysis is
enabled).

> It took me quite a while to find out why my programs failed as I did not
> expect such a (hidden) change.

It wasn't an intentional change for sure. However, the generated code
might have changed (hopefully it became more optimal) which simply exposed
some of these issues within the compiled code.

Actually, such issues are much easier to notice on other architectures,
aside from i386, because of the different calling convention and register
layout they use. I spent the better part of the last 3 years fixing code
which was originally intended for i386, to get it working better on ARM
(and others), where I ran into this exact "result remains uninitialized"
issue, along with many other "my code is good, fix your compiler" issues,
where it turned out the code was basically working accidentally, and it
was built on false assumptions...

Charlie___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

Re: [fpc-pascal] reference-counted function results not initialized to nil

[Sven Barth]

Am 25.06.2016 13:17 schrieb "Jürgen Hestermann" :
>
> Am 2016-06-25 um 09:23 schrieb Maciej Izak:
> > 2016-06-25 5:19 GMT+02:00 Michalis Kamburelis :
> > They were never guaranteed to be initialized to nil.
> > Result has special logic for string, dynamic array, method pointer and
variant (well documented ;) ):
> > "For a string, dynamic array, method pointer, or variant result, the
effects are the same as if the function result were declared as an
additional var parameter following the declared parameters. In other words,
the caller passes an additional 32-bit pointer that points to a variable in
which to return the function result."
> >
http://docwiki.embarcadero.com/RADStudio/Berlin/en/Program_Control#Handling_Function_Results
>
> Does that mean that we now need to read the whole documenation of Free
Pascal *and* also the whole documentation of Delphi?
> And what version of Delphi?
> Where is this documented within Free Pascal?

No one said that FPC's documentation is perfect, we do our best, but that
might still mean that there are missing pieces of information that we try
to fix as soon as they are known to be missing.

Regards,
Sven
___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

Re: [fpc-pascal] reference-counted function results not initialized to nil

[Jürgen Hestermann]

Am 2016-06-25 um 09:23 schrieb Maciej Izak:
> 2016-06-25 5:19 GMT+02:00 Michalis Kamburelis :
> They were never guaranteed to be initialized to nil.
> Result has special logic for string, dynamic array, method pointer and 
variant (well documented ;) ):
> "For a string, dynamic array, method pointer, or variant result, the effects are 
the same as if the function result were declared as an additional var parameter following 
the declared parameters. In other words, the caller passes an additional 32-bit pointer that 
points to a variable in which to return the function result."
> 
http://docwiki.embarcadero.com/RADStudio/Berlin/en/Program_Control#Handling_Function_Results

Does that mean that we now need to read the whole documenation of Free Pascal 
*and* also the whole documentation of Delphi?
And what version of Delphi?
Where is this documented within Free Pascal?

This has definitely changed with Free Pascal 3 as my programs suddenly did not 
work anymore and I had to add Setlength() all over my code.
Before we could rely on that managed types where always initialized (well, 
that's the purpose of managed types, no?).
It took me quite a while to find out why my programs failed as I did not expect 
such a (hidden) change.

___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

Re: [fpc-pascal] reference-counted function results not initialized to nil

[Sven Barth]

Am 25.06.2016 09:23 schrieb "Maciej Izak" :
>
> 2016-06-25 5:19 GMT+02:00 Michalis Kamburelis :
>>
>> They were never guaranteed to be initialized to nil.
>
>
> Result has special logic for string, dynamic array, method pointer and
variant (well documented ;) ):
>
> "For a string, dynamic array, method pointer, or variant result, the
effects are the same as if the function result were declared as an
additional var parameter following the declared parameters. In other words,
the caller passes an additional 32-bit pointer that points to a variable in
which to return the function result."
>
>
http://docwiki.embarcadero.com/RADStudio/Berlin/en/Program_Control#Handling_Function_Results

And them being var-parameters basically states that they retain whatever
value they had.

Regards,
Sven
___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

Re: [fpc-pascal] reference-counted function results not initialized to nil

[Maciej Izak]

2016-06-25 5:19 GMT+02:00 Michalis Kamburelis :

> They were never guaranteed to be initialized to nil.
>

Result has special logic for string, dynamic array, method pointer and
variant (well documented ;) ):

"For a string, dynamic array, method pointer, or variant result, the
effects are the same as if the function result were declared as an
additional var parameter following the declared parameters. In other words,
the caller passes an additional 32-bit pointer that points to a variable in
which to return the function result."

http://docwiki.embarcadero.com/RADStudio/Berlin/en/Program_Control#Handling_Function_Results

-- 
Best regards,
Maciej Izak
___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

Re: [fpc-pascal] reference-counted function results not initialized to nil

[David Emerson]

On 06/24/2016 08:19 PM, Michalis Kamburelis wrote:

After upgrading fpc 2.6.4 -> 3.0.0, I'm seeing a bug where (as noted in
subject) reference-counted function results are not being initialized to
nil.


They were never guaranteed to be initialized to nil.

Reference-counted types (unlike other types) cannot contain memory
garbage. But it doesn't mean that they are always initialized empty
when the function starts. You need to explicitly do Result := '' if
your code reads the Result later.

>
> See similar questions for Delphi:
> 
http://stackoverflow.com/questions/5336863/what-is-the-default-value-of-result-in-delphi
> 
http://stackoverflow.com/questions/5314918/do-i-need-to-setlength-a-dynamic-array-on-initialization/5315254#5315254


OUCH!!

I never dreamed that

local_var := some_function (param);

can result in the old value of local_var being used inside 
some_function. That's what's happening here. I fully expected the result 
dynamic array (embedded in a record type) to be initialized to nil, and 
have never taken care to initialize dynamic arrays and ansistrings. I'm 
astonished that this didn't bite me sooner, actually. Perhaps it is a 
culprit in occasional crashes.


I can't see any mention of this in the documentation:
http://www.freepascal.org/docs-html/ref/refse47.html
http://www.freepascal.org/docs-html/ref/refse89.html
http://www.freepascal.org/docs-html/ref/refse91.html


Luckily, FPC warns about it, at least in my simple test:


it also warns me in a simple test, but it doesn't give any warning in my 
not-simple program.



Is there any compiler option that I can use to force nil initialization 
of ref-counted function results?



Thanks,
David



___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

Re: [fpc-pascal] reference-counted function results not initialized to nil

[Michalis Kamburelis]

> After upgrading fpc 2.6.4 -> 3.0.0, I'm seeing a bug where (as noted in
> subject) reference-counted function results are not being initialized to
> nil.

They were never guaranteed to be initialized to nil.

Reference-counted types (unlike other types) cannot contain memory
garbage. But it doesn't mean that they are always initialized empty
when the function starts. You need to explicitly do Result := '' if
your code reads the Result later.

See similar questions for Delphi:
http://stackoverflow.com/questions/5336863/what-is-the-default-value-of-result-in-delphi
http://stackoverflow.com/questions/5314918/do-i-need-to-setlength-a-dynamic-array-on-initialization/5315254#5315254

Luckily, FPC warns about it, at least in my simple test:

$ fpc -vw a.lpr && ./a
a.lpr(5,20) Warning: function result variable of a managed type does
not seem to be initialized
blabla
blablablabla
blablablablablabla

Regards,
Michalis


a.lpr
Description: Binary data
___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal