Re: [fpc-devel] certain divisions in single precision only

[Martok]

Am 08.12.2019 um 18:41 schrieb Florian Klämpfl:

> As the delphi behavior is very i386/x86-64 specific, FPC deviates from 
> it (not to mention that the non-i386 behavior was implemented years 
> before a non-i386 delphi was available).

Jup, that's what I meant with "made runtime and compile time evaluation the
same". Without that change, cross and native compilers would have different
constant evaluation, which is obviously not good either.

Best,
Sebastian

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] certain divisions in single precision only

[Martok]

> Indeed, this is by design. See 
> https://wiki.freepascal.org/User_Changes_2.2.0#Floating_point_constants for
> how to explicitly specify a different precision.

That was long before the change Tomas discovered.

The gotcha here is about your fix to
, which together with the one you
linked made runtime and compile time evaluation the same, at the expense of
specifically breaking Delphi compat.

From the current Delphi Reference:

> In Delphi, intermediate results of Single precision floating-point
> expressions are always implicitly stored as Extended on x86.
> 
> By default, all x64 arithmetic operations and expressions involving only
> Single precision floating-point values retain high precision by storing
> intermediate results as Double precision values.

But since then, FPC sees Single / Int2Single = Single, with no high precision
intermediates.
There was either a bug or a thread here about the code that broke and some
issues with explicit casts also not helping, but I can't seem to locate either.


Best,

Sebastian

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] Progress on reviewing x86_64 optimizer overhaul and node semantic pass

[Martok]

Am 27.10.2019 um 17:48 schrieb Florian Klämpfl:
>> I guess the main difference is whether one prefers side-by-side
>> diffs or udiffs.
> 
> In particular partial commits as well as conflict resolution work much 
> better with TortoiseGit for me.

Oh yeah, conflict resolution is the thing nobody really gets right, but TGit is
a bit less wrong.
I've pretty much resigned myself to Ctrl-F ">"...

>> By the way, many people seem to use git on the client side... I remember 
>> there
>> were talks about moving the main repo to git. What happened to that?
> 
> Boring job nobody wants to do?

Haha, thought so :D


Best,

Sebastian

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] Progress on reviewing x86_64 optimizer overhaul and node semantic pass

[Martok]

> cover one single topic. Today, using e.g. TortoiseGit on Windows (sorry, 
> on Linux there is no tool which comes close) such patches can be 
> re-arranged without too much hazzle.

Just plain ol' git-gui can also do it. SmartGit is cross-platform and also
pretty nice. I guess the main difference is whether one prefers side-by-side
diffs or udiffs.


By the way, many people seem to use git on the client side... I remember there
were talks about moving the main repo to git. What happened to that?


Best,

Sebastian

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] Minor debate with ISO standard on case blocks

[Martok]

ue is that FPC won't compile some well-known ISO Pascal
> code <https://bugs.freepascal.org/view.php?id=35859> because some of the case
> blocks don't exhaustively cover all inputs (which itself might be a sign of 
> bad
> programming, but could be justified if said values are determined to be
> logically impossible, in which case the else blocks should contain assertions 
> or
> internal errors).
> 
> Just interpreting the standard, I think that the error should be run-time, not
> compile-time (although definitely keep the warning).  That's just my take on 
> it
> though.  If it is to be changed, it should be simple enough by configuring the
> 'elselabel' field to point to an error-raising routine rather than 'endlabel'
> (which occurs if there's no else block).
> 
> Gareth aka. Kit
> 
> 
> <https://www.avast.com/sig-email?utm_medium=email_source=link_campaign=sig-email_content=emailclient>
>   Virus-free. www.avast.com
> <https://www.avast.com/sig-email?utm_medium=email_source=link_campaign=sig-email_content=emailclient>
> 
> 
> <#DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2>
> 
> 
> ___
> fpc-devel maillist  -  fpc-devel@lists.freepascal.org
> https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
> 


-- 
Regards,
Martok

Ceterum censeo b32079 esse sanandam.

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] [Suggestion] Enumeration range-check intrinsic

[Martok]

> If you want to nitpick, the compiler will perform 1/2/4 byte writes for
> enums of those sizes, so the full reserved data is in fact
> used/initialised. Again: the only relevant part in this discussion is
> the valid values. The reserved/used/accessed/written/... storage size is
> unrelated to that to the extent explained in my previous message.
That was exactly my point!

The storage size is unrelated to the "used"/"valid" size, therefore there is
little point in being able to set it.
I can tell the compiler "this type is 4 bytes wide", but it will still only ever
consider the lowest byte if that's how many elements there are.

> This inconsistency would mean that we at least need two versions of
> defutils.getrange() in the compiler: one for (some) range checks (using
> the named range), and one for other reasoning (which must use the
> basedef range). There may be other such places that need modifications.
Interestingly, that's not the case, I checked.
There are like 2 major places that need adjustment, everything else already
correctly uses getrange for the type range and min/max for label range. It
almost looks as if most of the compiler was written assuming these two are not
the same.

The JVM class generator is the only real challenge. These days, one could
probably take a look at .Net Core to see how a managed language does this.

>> Arrays of such a type should not be allowed (as for enums with jumps right 
>> now)
>> (* although they could be without surprises, Array[tmyenum] would simply be
>> Array[Byte] *).
> 
> The whole point of a directive/modifier would be to get
> Delphi-compatible enums, not to add a third variant.
Where do you see a third variant? If you think this is anything but a
"Hejlsberg-Enum", I seriously failed to explain it.

The proposed syntax would be a shorthand for what Delphi does when you write
  {$Zn} tenum = (ea,eb,ec,ed);
With n:=sizeof(basetype).

Nothing else, nothing more.

> I meant that Delphi's range checking for enums/subranges doesn't make
> any sense due to the inconsistency between the "valid" and "declared" range.
The thing about $R and Delphi: a) it is purely ancillary and not part of the
type system. A *lot* of stuff is not rangechecked there at all. This has not a
lot to do with the nesting of type ranges.
b) only pred/succ, literal assignment without hard cast and assignment from
larger to smaller type are guaranteed to check if a subrange type is exceeded by
an action. Nothing else does - anything(!!) that fits in the same storage is a
valid value.

Both of that is documented behaviour, and has been for decades.


> E.g., take this Delphi program (tested with Kylix 3; YMMV with modern
> Delphi versions, to which I don't have access)
> [snip]
> This is both 100% inconsistent and 100% logical.
> On the other hand, you have this whole inconsistency where 255 is both a
> valid and an invalid value. So when a range check happens to be
> inserted, you get a range check error, and when it isn't, you don't.

One might argue that it is 100% consistent as well. Compile in {$R-} (remember,
in TP and Delphi it's a debugging aid, not a
thing-you-have-to-do-to-get-somehwat-safe-results like in FPC), and only the out
of bounds array accesses remain. And those are 100% equivalent to our old
friend, the unchecked jumptable access.

> As described in that post, the issue is that the default FPC units
> declare various enumeration types without any specific modifiers. Even
> if all default FPC units would be changed to Delphi-compatible enums,
> then you would still get issues when mixing and matching units declared
> with differently declared enumeration types, since there is no obvious
> way to know how a particular enumeration type has been declared.
Except, you know, looking at the source?

That's like arguing that there is no way to know if a record is packed or not.
Of course there is, it's a keyword right in the declaration.

Or, actually, more like asking what the calling convention of a function could
be. Usually you don't need to know, but when you do, it's either a keyword at
declaration site, or a {$CALLING} setting is in effect. No ambiguity at all.

That's why I say the decision must be clear at declaration point, not for every
access (as was previously suggested).


Best,
Sebastian





___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] [Suggestion] Enumeration range-check intrinsic

[Martok]

Am 14.07.2019 um 14:18 schrieb Jonas Maebe:
>> Side note: if this was done 100% consistently (and it does make sense!), the
>> PACKENUM directive would be completely useless. There is no point in being 
>> able
>> to specify the storage size of an enum when it can be and is ignored at will.
> 
> In other cases, the requested storage size is reserved.

Oh please, don't troll. I know that you know that "is reserved" and "is used"
are not the same thing.

> The most common cases to specify a storage size would be alignment, and
> the ability to add more valid values in the future without having to
> change the layout of a data structure.
... the only reasonable application of which is some kind of IO or an ABI that
shouldn't change, for which enums must not be used directly in the first place.

> It would solve nothing, and rather demonstrates the disconnect that
> apparently still exists in this discussion: the ability to bitpack enums
> (and hence reduce their storage size) is a mere side effect of the fact
> that their valid range only goes from the lowest to the highest declared
> value. You could reserve 4KB for an enum
No, you couldn't. The largest ordinal type is QWORD.


> The issue with a modifier or directive that would change the definition
> of which values are valid for an enumerations with this modifier/within
> this directive, is (besides the inconsistent low/high and range checking
> behaviour from Delphi this would require,
This is easy, enums already have separate min/max and a basedef. Make the
basedef a type that can hold an orddef, modify calcsavesize and packedbitsize
accordingly, done.
Low/High will continue to provide the named range.
Arrays of such a type should not be allowed (as for enums with jumps right now)
(* although they could be without surprises, Array[tmyenum] would simply be
Array[Byte] *).

On Subranges, that's also simple and obvious. That's the beauty of it, explicit
sizes would be explicit. They are not "inherited":

  tmyenum : Byte = (ea, eb, ec);
  tsubenum = eb..ec;
 //^ basedef=tmyenum, no explicit type, "fpc-style" enum,
 //  inherited symtable with ordinals $01 and $02
  tsubwenum: Word = eb..ec;
 //^ basedef=u16inttype, new symtable with ordinals $0001 and $0002

While the first example looks inconsistent, it is the same as what we already 
have:
{$PACKENUM 1}
  tmyenum = (ea, eb, ec);
{$PACKENUM DEFAULT}
  tsubenum = eb..ec;
tsubenum will be 4-bytes, tmyenum is 1 byte, same as with the new syntax.
tsubenum remains assignment compatible to tmyenum.

In contrast tmyenum.eb and tsubwenum.eb are not compatible, as the obviously
cannot be, since they have different _explicit_ sizes. I can't think of a
usecase for the second declaration in the first place, but getting correct
behaviour as a side effect is always a good sign.


> the post I linked near the beginning of
> the thread:
> https://forum.lazarus.freepascal.org/index.php/topic,45507.msg322059.html#msg322059

As I wrote in the last message, those points would be fully addressed by this
proposal. Choosing at declaration time is the only valid solution, all modes
will behave exactly the same once the type is defined.


Best,

Martok


___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] [Suggestion] Enumeration range-check intrinsic

[Martok]

> In Delphi's type system (and in C, C++, and C#), the valid values for an
> enum are always all values that fit within its storage size, and that
> storage size is fixed.
> 
> In FPC's type system (and in Ada and Java), the valid values for an an
> enum are only the values between the lowest and the highest declared
> enum element (in Java it's even more strict, in that when you have
> "enums with holes", the holes are also invalid), regardless of the
> storage size. Moreover, this storage size can vary depending on whether
> or not the item is in bitpacked storage.

Side note: if this was done 100% consistently (and it does make sense!), the
PACKENUM directive would be completely useless. There is no point in being able
to specify the storage size of an enum when it can be and is ignored at will.


C# and C++ do the same with explicitly sized enumerations. Who would I have to
bribe to get that in FPC?

{$mode objfpc}
type
  tmyenum : Byte = (ea, eb, ec);

That would solve literally all problems we ever discussed:
 - the "scoped constant" concept is obvious when used
 - for bitpacking purposes, the bitsize is that of the explicit ordinal
 - therefore, any same-size cast (API return, file io, network io) is valid
 - {$Mode TP} and {$Mode Delphi} would treat all enum declarations as sized
enumerations of whatever $Z is set to at declaration time, fixing the "same
syntax but invisibly different result" portability issue
 - both type concepts would be consistently handled in the same type system
 - whenever that syntax is not used, all existing optimizations are valid
 - when it is used, others may be chosen
 - the case label checks would also catch any problems we don't yet see


Anything else (including the intrinsics this is about) is just a hacky
workaround to the difference Jonas summarised above, IMHO.

-- 
Regards,
Martok

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] [Suggestion] Enumeration range-check intrinsic

[Martok]

Am 06.07.2019 um 09:01 schrieb Ondrej Pokorny:
> Ord(aEnum) for invalid enumeration values is undefined ;)

If there was any logic here, it should be, but it's not ;-)

The documentation page specifically mentions Ord as the older syntax to hard
casts. Whatever it contains, any enum is always smaller or equal to the largest
possible bitsize, which is Longint.


Alternatively, which should be completely equivalent but is not because Ord is
Delphi-compatible and subranges are not:

generic function TStreamHelper.ReadEnum(out aEnum: T): Boolean;
var
  tmp: Ord(Low(T))..Ord(High(T)) absolute aEnum;
begin
   if Read(aEnum, SizeOf(aEnum)) <> SizeOf(aEnum) then
 Exit(False)
   else begin
 Result := (tmp >= Ord(Low(T))) and (tmp <= Ord(High(T)));
   end;
end;

This *could* be optimized out by FPC, but not by a Borland compiler.


-- 
Regards,
Martok

Ceterum censeo b32079 esse sanandam.

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] [Suggestion] Enumeration range-check intrinsic

[Martok]

Am 06.07.2019 um 01:21 schrieb J. Gareth Moreton:
> According to this <https://www.freepascal.org/docs-html/rtl/system/ord.html>, 
> a
> Longint.
Ah, right.
I checked the source again: that is true only if the argument is an enum. If it
is any other ordinal type or Pointer, the result is of the same word size (but
possibly different signedness) as the source.

In that case, unlike what was previously said in this thread, this should simply
work:

generic function TStreamHelper.ReadEnum(out aEnum: T): Boolean;
begin
  if Read(aEnum, SizeOf(aEnum)) <> SizeOf(aEnum) then
Exit(False)
  else begin
Result := (Ord(aEnum) >= Ord(Low(T))) and (Ord(aEnum) <= Ord(High(T)));
  end;
end;

So this is a comparison of Longints (add a cast to Longword to fix the overflow
bug), nothing that should be optimized out. This should also work for subranges
in their host types. Unless the optimizer removes the typeconvnodes...


Of course, this needs to be done everywhere a subrange is passed anywhere that
is not FPC, so it can't really fix anything.


-- 
Regards,
Martok


___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] [Suggestion] Enumeration range-check intrinsic

[Martok]

I think there is a shorter version, but what is the exact return type of Ord()
again?

By the way, what you're reinventing here is type marshalling, for one single
type class, and in a very roundabout way.
If that is really what the discussion settled on, it would probably make sense
to not tack this onto a particular stream implementation but instead provide a
general marshaller for it.

-- 
Regards,
Martok


___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] [Suggestion] Enumeration range-check intrinsic

[Martok]

> But I will not argue the point, it is largely a matter of personal 
> interpretation
> of the intended use of an enumerated.

Well, yes. Since any argument from authority is discarded, what remains is
personal interpretation.


> But I will definitely adapt the FPC documentation to be more clear about the 
> actual
> behaviour of the compiler.
And this is why we can't have nice things.

Please don't get me wrong (really!), this is nothing against your work
specifically, but that the documentation has to follow the implementation and
the implementation follows no particular philosophy at all is what got this
interpretation of the language into this mess in the first place.


FPC is in dire need of a language specification effort. But I don't think anyone
who would be willing to drive it cares enough at that point.


-- 
Regards,
Martok


___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] [Suggestion] Enumeration range-check intrinsic

[Martok]

Am 05.07.2019 um 11:28 schrieb Michael Van Canneyt:
> But IMO such a strict stance is untenable: you cannot ensure correct values 
> in all cases even within the confines of the language (as demonstrated 
> before).

Correct. Even if certain (exactly two) people here believe differently.


> So, in my book, an intrinsic to check for a correct value in the places where 
> an 
> invalid value can enter the system is a welcome language feature.

In my book it's absolutely insane that this is needed *just because* FPC wants
to be different, but yes, it is required.


-- 
Regards,
Martok

Ceterum censeo b32079 esse sanandam.

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] [Suggestion] Enumeration range-check intrinsic

[Martok]

Am 05.07.2019 um 12:20 schrieb Ondrej Pokorny:
>> Anything that even looks like a subrange cannot be used for these interfaces.
> 
> You may use enumerations for these interfaces - you may just not fill 
> them directly but through an integer variable in between when reading 
> from outside.

Then you're not using them *in* the interface ;-)

-- 
Regards,
Martok

Ceterum censeo b32079 esse sanandam.

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] [Suggestion] Enumeration range-check intrinsic

[Martok]

Am 05.07.2019 um 14:24 schrieb Sven Barth via fpc-devel:
> Huh? The comment regarding dynamic packages is nonsense. The compiler and RTL
> check that the checksums of the involved interface sections match and thus 
> there
> is no problem there. 

Does that cascade? If so, that's better than Delphi, nice :)


-- 
Regards,
Martok

Ceterum censeo b32079 esse sanandam.

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] [Suggestion] Enumeration range-check intrinsic

[Martok]

Am 05.07.2019 um 02:52 schrieb Michael Van Canneyt:

> With this sentence you forbid storing or communicating enumerated values in 
> any way: 
> file, database, over network. It can be used only in a computer program and 
> never
> leave the context of the running program under any form. Because as soon as
> it is somehow communicated, there is a chance it becomes invalid in return
> communication.

This is the conclusion from Jonas' arguments, which he explicitly confirmed when
we had the discussion two+ years ago.

You're also missing another important type of 'communication': calling non-FPC
APIs or even just FPC libraries of possibly different version (see: Run-Time
Packages!)

Anything that even looks like a subrange cannot be used for these interfaces.

-- 
Regards,
Martok

Ceterum censeo b32079 esse sanandam.

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] [Suggestion] Enumeration range-check intrinsic

[Martok]

Am 05.07.2019 um 10:47 schrieb Michael Van Canneyt:
> Note that the explanation is IMO fuzzy, not to say contradictory:
> 
> 'An enumerated type defines an ordered set of values by simply listing 
> identifiers that denote these values'
> 
> is at odds with (under the heading 'Enumerated Types with Explicitly Assigned 
> Ordinality'):
> 
> 'An enumerated type is, in effect, a subrange'

No contradiction here: the corresponding implied subrange goes from lowest to
highest assigned value, which is 0..n for dense enumerations and
low(consts)..high(consts) for Enumerated Types with Explicitly Assigned
Ordinality. There is nothing special about the holes, they just have no names.
This is also how the Delphi debugger displays them.


Again, I recommend reading the TP5 and TP7 books, which contained the
*rationale* for most of the type system. That got left out when they rewrote the
content for Delphi Help.


-- 
Regards,
Martok

Ceterum censeo b32079 esse sanandam.

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] [Suggestion] Enumeration range-check intrinsic

[Martok]

Am 02.07.2019 um 20:54 schrieb Ondrej Pokorny:
> The conversation is more than a year old and I already posted a patch - 
> also more than a year ago. I haven't received any feedback from the 
> compiler developers since I uploaded the patch. Maybe you will have more 
> luck.
FWIW: I've been flying this patch in the snapshot builds ever since, and nobody
complained. So it's probably not immediately broken ;-)

The problem is much bigger than enumerated types though, all subrange types are
fundamentally incompatible between FPC and Borland. I don't remember if your
patch handles those correctly as well? That probably needs a few test cases?

-- 
Regards,
Martok

Ceterum censeo b32079 esse sanandam.

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] When will the next version of FPC be released?

[Martok]

>> Really ?
>> I wonder what kind of code you write to make this 'super buggy' statement.
>>
>> I myself write lots of code, and never encountered an issue.
>> But then, I normally don't use dubious constructs to begin with.

> In fact, we already have several bug reports related to overloading/generics.
> 
> See 
> https://bugs.freepascal.org/view.php?id=27690 
> https://bugs.freepascal.org/view.php?id=30776
> https://bugs.freepascal.org/view.php?id=28824

Some more non-generic-related ones, just ones I can remember right now:
https://bugs.freepascal.org/view.php?id=35580
https://bugs.freepascal.org/view.php?id=31215 (don't mind the symptom, the cause
is a typeconv issue)
https://bugs.freepascal.org/view.php?id=35573


What bugs me most is how one small fix (such as r42042) breaks other things that
should be orthogonal but are not.

All because there are at least four partially recursive resolution methods
following slightly different rules, all for the same question "given the passed
argument defs, what is the path of least conversion to something of matching
name in the current symtable?".
But yeah, that's just core having a different opinion of code duplication than 
I do.

-- 
Regards,
Martok


___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] When will the next version of FPC be released?

[Martok]

> What with overloads ?
> 
> Procedure MyProc(a:T);
> Procedure MyProc(a : TObject);
> 
> How do I know which one will be called ?

Once again: FPC needs a defined method resolution order, *and stick with it*.

The current ad-hoc inheritance/typeconv/:= overload/overload/operator overload
mess is already completely unpredictable and super buggy.

Based on a specificity approach, the answer to your question would be obvious:
The latter if a is (a descendant of) TObject, a new specialization otherwise.


-- 
Regards,
Martok


___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] Compiler picks wrong overload

[Martok]

> More broadly I think the bigger problem is FPC's internal over-eagerness to
> implicitly cast things to other things... for example, if you have a type with
> an ":=" operator overload (or as Delphi calls it, an "Implicit" operator
> overload) from both "Pointer" and "TObject", FPC will always pick the 
> "Pointer"
> one even if given an explicit TObject variable. They (currently) cannot
> co-exist. But certainly, they should be able to, if you ask me.

I still find "nil->Pointer->dynarray->UCS4String" the most hilarious "weight 1"
autoconversion (see 0031215, which is also an overload bug).


-- 
Regards,
Martok


___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] "Case statement does not handle all possible cases" Warning

[Martok]

It's not DFA.

See my proposal thread from January and Jonas' implementation from this week:

<http://lists.freepascal.org/pipermail/fpc-devel/2019-January/039972.html>
<http://lists.freepascal.org/pipermail/fpc-devel/2019-May/040737.html>

> Warnings should indicate the possibility of unstable code due to 
> uninitialised values

As long as the core question remains unanswered, these are required exactly
because in cases (pun intended) where they exist, the compiler may do something
very different from what was intended, and even be wildly different between
platforms.

I would prefer literally any other solution (such as the two-line-patch my
Windows trunk builds have contained for two years now), but this is the one that
gets merged. Go figure.


-- 
Regards,
Martok

Ceterum censeo b32079 esse sanandam.

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] [Patch/RFC] Warnings for (in/over)complete case statements

[Martok]

Hi,

don't get me wrong, I like having something of mine included, but I'm genuinely
surprised you used it in the way I designed it.

As you said, the warning and code that is actually generated must agree,
otherwise they're useless. And there is a huge gap in the definition of the
Freepascal interpretation of what "all values of the type" even means, with the
code-as-written sometimes using one interpretation and sometimes the other.

On the topic of enums with assigned ordinals:

> I simply forgot to take them into account. I'm tempted to disable the 
> warning altogether for them, because
Please don't. Having sanity checks for "easy" types (where you don't really need
them) and not having them for "complicated" types (where they would likely catch
programmer errors) is not super helpful, and I would consider this as *worse*
than the status quo of not having warnings at all, because it would give a false
sense of safety ("My code compiles with -Sew? I must have done everything 
right!").
I'll never understand why Borland took that short cut for enumeration typeinfo
and got away with it.

> 1) Semantically, it would make sense to give a warning only if not all 
> explicitly defined enum values are covered
Agreed.

> 2) However, you still can't give an "unreachable code" warning if 
> there's an "else" in that case, because implicit values can also be 
> stored in the enum
Which takes us aall the way back to the original problem.

In my (and Borlands, and Microsofts) opinion, this program should compile
without raising the two new warnings (all named values are used AND else for
unnamed values 3 and 4), rangecheck as correct (3 in [2..5]), and be fully
defined at runtime. DFA should show that the else branch is always taken, which
may then mark the case labels as dead code.

program Project1;
{$mode objfpc}
type
  TMyEnum = (one = 2, two = 5);
var
  A: TMyEnum;
begin
  A := TMyEnum(3);
  case A of
one: Exit;
two: Exit;
  else
Writeln('Other');
  end;
end.

With R := Default(TMyRange), which assigns 0 [side note: why is the default
value an invalid value? That makes no sense at all], I would expect the same
results except for a range check warning at compile time (Range check error
while evaluating constants (0 must be between 2 and 5)).

I would *wish for* (and Borland documented this) the code to still be
runtime-safe even in $R- in this case, but I'm well aware that you and Florian
have already declined that. For completeness: subrange types silently upgrade to
their storage types if required, which to me implies also in comparisons, which
implies the "none of the above" label must always mean the entire storage type.


I'm still in favour of making an effort to formally define at least the core
elements of the Freepascal languages (syntax, statements, types, overloads,
resolution order, typeconv order, ...) with as few implementation quirks as
possible and then align the compiler to it (not the other way around). Having
less corner cases to juggle in one's mind may make things easier, and a formal
spec is also a good reference for future extensions to see if ideas are
orthogonal or not. But that's a whole different topic.


-- 
Regards,
Martok

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] fpcmkcfg and fpc.cfg questions

[Martok]

Am 25.03.2019 um 14:43 schrieb Joost van der Sluis:
> When you discover things like this, it's better to report it. That way 
> you avoid double work for others with the same problem.
That piece was from way back when the footnote statement of "Don’t write about
Laz/FPC-Bugs, I’m not involved in development there." was actually true and I
haven't looked at it since, because it "just works" ;-)

Although, that *is* true again, so it's 50/50 if I would report it today.

-- 
Regards,
Martok


___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] fpcmkcfg and fpc.cfg questions

[Martok]

Hi,

this is what I supply with the snapshot builds:

https://pastebin.com/KseAwBAM

The multicall binary lives in
  $(LazarusDir)\fpc\$FPCVer\bin\$TargetCPU-$TargetOS\fpc.exe
This script is in
  $(LazarusDir)\makeconfig.cmd

You'll notice this is the default Windows Installer path structure and *will not
currently work with fppkg*. But you will get a working compiler.

I used '-o "%CD%\fpc.cfg"' exactly because fpcmkcfg does not handle relative
paths well.

--
Regards,
Martok

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] TRegistry and Unicode

[Martok]

Am 26.02.2019 um 19:12 schrieb Bart:
> This leaves my initial "itch": input strings are CP_ACP (so can be
> anything), output strings are CP_UTF8 always.
Unless your DefaultSystemCodePage is CP_UTF8 (in which case UTF8Encode is not
needed), the next action on the string will convert away from UTF8 anyways.

String codepages are not stable, there is almost never a point in explicitly
setting one...

-- 
Regards,
Martok


___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] Wrong floating point Currency calculations

[Martok]

> Is this a known bug?
> 
> (Btw. I tested on win32.)

Maybe this?
https://bugs.freepascal.org/view.php?id=33963

-- 
Regards,
Martok

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] Fppkg

[Martok]

> Do people actually use fppkg for anything?
Well, *I* dont't know anyone, and the recent stuffs certainly isn't any form of
incentive.

But apparently, fppkg is required to discover fpmake packages that fpc already
knows about, because... reasons?


(Disclaimer: I detest any form of package manager, so my opinions very much
*should* be void.)

-- 
Regards,
Martok


___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] Fppkg

[Martok]

Am 04.02.2019 um 10:43 schrieb Alfred:
> Would it be possible to add (feature request) a local fppkg.cfg (like fpc.cgf)
> that contains these defaults paths, so they can be modified and used by
> the fppkg executable ? To distinguish between different installs.

May I add that ideally the Lazarus packager-fppkg interface would use the
fppkg.cfg local to $(CompPath). With a proper setup, one could make sure fpc.cfg
and fppkg-config do match - currently they will only agree for the compiler of
the first-ever fppkg.exe invocation on a machine (the one that creates
%LocalAppdata%\Freepascal\fppkg\config), hence break any time the compiler path
changes.

-- 
Regards,
Martok


___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] Sorry for poor testing

[Martok]

Am 14.01.2019 um 15:01 schrieb J. Gareth Moreton:
> Martok mentioned doing some checks differently in the bug report in question,
> such as 6 comparisons being faster than a jump table.  Are there any others
> worth mentioning?
Not neccessarily faster, but in that code definitely smaller. Is there a way to
directly estimate how large the generated code might be, maybe something like
"numlabels*constant"?

For the "faster" question, the cache line occupation matters more than
instruction throughput. If your jumptable gets large enough to force a full data
cache line flush, you could have done a lot of instructions in the time the CPU
waits for the data. And I haven't found reliable information how the
Spectre-related microcode updates changed that, given that both cmp/jmp and
jmp[indirect] used to be heavily optimized, but were both affected.


-- 
Regards,
Martok


___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] [Patch/RFC] Warnings for (in/over)complete case statements

[Martok]

Am 02.01.2019 um 11:30 schrieb Florian Klämpfl:
> The compiler is for speed reasons not compiled with $R+/$O+, so the safety 
> else clause have its purpose, but they should
> throw an internal error instead.
And here we are at that other issue again. There is absolutely no guarantee the
safety clause is even present in the compiled ppc. The improved jumptable
generation actually makes it less likely! (the state of $R,$O doesn't change 
this)

So there's really 2 possible goals:
- safety against programmer error: a new stringtype is invented, and not
considered here. The coverage warning will tell the programmer, just as an IE
(maybe) would during testing -- it's a bug either way.
- safety against data error: def deserialized from damaged ppu? Undefined
behaviour, the else clause may not be present in the compiler binary, since all
legal values are handled. Which is exactly what the Unreachable Code warning
would tell you.


This segment of the compiler is specifically the kind of scenario that these
warnings are designed to catch, and also the reason why they are warnings, not
hints. Code that someone wrote and language spec do not agree, and the compiler
will do something that is *very likely* unintended.


-- 
Regards,
Martok


___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] [Patch/RFC] Warnings for (in/over)complete case statements

[Martok]

Am 02.01.2019 um 11:19 schrieb Michael Van Canneyt:

> Consider the following:
> 
> Type
>TMyClass = class
>Private
>  function GetString(AIndex: Integer): string;
>Published
>  Property MyString : String Index 1 Read GetString;
>end;
> 
> function TMyClass.GetString(AIndex: Integer): string;
> 
> begin
>case AIndex of
> 1 : Result:=GenerateSomestringvalue;
>end;
> end;
> 
> I don't think there should be errors or warnings.

Good example.

Although... you *could* call GetString from some other class member function,
and at that point Result would be undefined. I don't think the compiler can
prove that?
Wait - why does that code not raise a 5033 "Function Result does not seem to be
set"? Add a second property "Property MyOtherString : String Index 2 Read
GetString;", and now its provably wrong. No warning. This seems wrong,
considering what we just talked about on fpc-pascal.

-- 
Regards,
Martok


___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] [Patch/RFC] Warnings for (in/over)complete case statements

[Martok]

Am 02.01.2019 um 11:06 schrieb Bart:
> So now I will have to add a useless else statement for every case
> statement that uses e.g. integers,
Well, as Jonas argues on fpc-pascal, you are supposed to add useless statements
anyway, which the compiler will then 100% remove...

j/k.

In all seriousness though, that is why I didn't include all integers. But enums
and small ordinals are often used as a "what is this data" tag, where it is
usually an oversight if some value is not checked.
I'm already not super happy this also applies to Char (a OS_8 ordinal), because
it makes case-based regular language parsing quite noisy.


-- 
Regards,
Martok


___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

[fpc-devel] [Patch/RFC] Warnings for (in/over)complete case statements

[Martok]

Hi all,

The attached patch adds two messages inspired by C compiler's -Wswitch-enum and
-Wcovered-switch-default. Building on the recent label count refactoring, this
was fairly straightforward.

- If a case statement on an ordinal does not contain labels for all values of
the ordinal, and no else statement is given, raise a new warning (W6059). This
is actually defined as an error in ISO7185 and a dynamic-violation in IEC10206.

- If a case statement has labels for the entire range of the type, and an (now
never used) else statement is present, a warning "Unreachable code" is raised.

Both cases are clearly something where the compiler and programmer don't agree
on something, hence drawing attention to these situations.

The checks are enabled only for enumerated types and small (1-byte) integers. In
C, they are only for enumerated types, I added small types because they are
often used as tag types, where this check is extra useful.

Now, the RFC part. I have a few open questions...

* does it make sense to do that to all integral types? In ISO mode, it's
formally required for all case statements. I left it out for now as sometimes
case..of is used as a shorthand for multiple in..[range], there would be more
detections that look like false positives.

* in ISO mode, there are 2 solutions: in Standard Pascal, not covering the
entire type is a compile-time error. In Extended Pascal, this is a runtime
error. Which one is followed usually?

* Building the compiler itself cycles with -Sew, so it bails on the first
occurrence of one of these issues - turns out they're all over the place.
However... I think the warnings are correct and expose imperfect code here.
Take this example (one of the first encountered):
<https://github.com/graemeg/freepascal/blob/master/compiler/defutil.pas#L1630>
stringtype is an enum and all values are used. There cannot be any other values
(as we have established last year). Therefore, the else clause is just as wrong
as an "if false then", which would be caught already.

* Adding to the previous, since it is now possible to discover forgotten items
or later additions by the other warning, removing these 'safety' else-clauses
would improve code quality.


What do you think?

-- 
Regards,
Martok

From e32addb6583c8b752c168fe221385566499625bb Mon Sep 17 00:00:00 2001
From: Martok 
Date: Tue, 1 Jan 2019 18:59:56 +0100
Subject: [PATCH] Report warnings for incomplete and over-complete case
 statements

+ regenerated messages
---
 compiler/msg/errore.msg |   5 +-
 compiler/msgidx.inc |   5 +-
 compiler/msgtxt.inc | 816 
 compiler/ncgset.pas |  28 +-
 4 files changed, 440 insertions(+), 414 deletions(-)

diff --git a/compiler/msg/errore.msg b/compiler/msg/errore.msg
index b6448c25ba..449a9a1a4c 100644
--- a/compiler/msg/errore.msg
+++ b/compiler/msg/errore.msg
@@ -2350,7 +2350,7 @@ sym_e_type_must_be_rec_or_object=05098_E_Record or object 
type expected
 #
 # Codegenerator
 #
-# 06049 is the last used one
+# 06059 is the last used one
 #
 % \section{Code generator messages}
 % This section lists all messages that can be displayed if the code
@@ -2505,6 +2505,9 @@ cg_n_no_inline=06058_N_Call to subroutine "$1" marked as 
inline is not inlined
 % The directive inline is only a hint to the compiler. Sometimes the compiler 
ignores this hint, a subroutine
 % marked as inline is not inlined. In this case, this hint is given. Compiling 
with \var{-vd} might result in more information why
 % the directive inline is ignored.
+cg_w_case_incomplete=06059_W_Case statement has unhandled operand values
+% The case statement does not contain labels for all possible values of the 
operand, but not else statement is used.
+%
 %
 % \end{description}
 # EndOfTeX
diff --git a/compiler/msgidx.inc b/compiler/msgidx.inc
index 7100a9eee8..47752cfeb4 100644
--- a/compiler/msgidx.inc
+++ b/compiler/msgidx.inc
@@ -696,6 +696,7 @@ const
   cg_e_function_not_support_by_selected_instruction_set=06056;
   cg_f_max_units_reached=06057;
   cg_n_no_inline=06058;
+  cg_w_case_incomplete=06059;
   asmr_d_start_reading=07000;
   asmr_d_finish_reading=07001;
   asmr_e_none_label_contain_at=07002;
@@ -1106,9 +1107,9 @@ const
   option_info=11024;
   option_help_pages=11025;
 
-  MsgTxtSize = 82706;
+  MsgTxtSize = 82758;
 
   MsgIdxMax : array[1..20] of longint=(
-28,106,349,126,99,59,142,34,221,67,
+28,106,349,126,99,60,142,34,221,67,
 62,20,30,1,1,1,1,1,1,1
   );
diff --git a/compiler/msgtxt.inc b/compiler/msgtxt.inc
index baf2592f67..7ee50d0a45 100644
--- a/compiler/msgtxt.inc
+++ b/compiler/msgtxt.inc
@@ -869,493 +869,500 @@ const msgtxt : array[0..000344,1..240] of char=(
   'ion set: $1'#000+
   '06057_F_Maximum number of units ($1) reached for the current target'#000+
   '06058_N_Call to subroutine "$1" marked as inline is not inlined'#000+
-  '07000_DL_Starting $1 styled assembler parsing'#000+
-  '07001_D

Re: [fpc-devel] TDef mode question

[Martok]

Am 27.12.2018 um 20:48 schrieb Jonas Maebe:
> No, there is not. If a def needs to be different depending on the 
> language mode, you have to store it in the def.
Thought so. Thank you for confirming!


Cheers,
Martok

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

[fpc-devel] TDef mode question

[Martok]

Hi all,

just a very quick question - I think the answer is no, but that doesn't mean
anything:

Is there a (ppu-persisted) way to find out in what language mode (fpc, iso,
tp..) a tdef was originally declared? The current_settings.modeswitches
obviously refer to the module it is currently used in.


Cheers,
Martok

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] Case block optimisation

[Martok]

Am 15.12.2018 um 18:08 schrieb J. Gareth Moreton:
> So here's something else I've been playing with in the interim... I've been
> working on improving the algorithms used when compiling case blocks.  It was
> driven partly by my binary search idea for certain kinds of case block, which 
> I
> wrote up over here: http://wiki.freepascal.org/Case_Compiler_Optimization

Feel free to include '0033093_Casenode_order.patch' from
<https://bugs.freepascal.org/view.php?id=33093>
It's actually already a bit simpler in your version ;-)

The one thing that still bugs me (but I know it's not really seen as a problem
in FPC) is that every platform cg theoretically has to implement all of that,
causing a lot of duplication.

I like the use of more expressive intermediate names. But there was a lot of
overflowed-aint-shuffling, are you sure that still works everywhere?

Bug 0032115 provides a "nice" test case for things that can go wrong with
different word sizes, and is also a good test for the true label count.


-- 
Regards,
Martok

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] x86_64 Optimizer Overhaul

[Martok]

Sorry for hijacking the thread. Your mail client issue makes the conversation
really hard to follow, so I have literally no idea what the current subtopic of
a reply chain is, and there's little point in properly detaching a thread.


Am 15.12.2018 um 18:13 schrieb J. Gareth Moreton:
> I dare ask, does that mean we should avoid workarounds in the compiler (and
> our own programs) that aim to avoid constant construction and destruction of 
> objects, and instead try to improve the memory manager?

I was thinking more along the lines of avoiding cycle-counting special paths at
the cost of reliability, when there are much larger issues that would benefit
every program.

I would not be surprised if some of the large difference Simon listed when
calling out the bounty come from this side, instead of raw instruction 
throughput.


> Thus, I would imagine that Delphi's *default* internal memory management
> system is more along the lines of what is done in FPC's cmem unit, which is
> well known to be objectively faster than FPC's default memory manager
I'm fairly certain the runtime is written in Pascal, except for parts of the
startup code. The memory manager at some point (I think D2006?) adopted FastMM:
<http://docwiki.embarcadero.com/RADStudio/2010/en/Configuring_the_Memory_Manager>
In any case, FPC's cmem on Win32 calls into mscvrt, and that is so slow that I
killed the test code after a couple of minutes, where even FPC-builtin was done
after 10 seconds.



-- 
Regards,
Martok


___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] x86_64 Optimizer Overhaul

[Martok]

Am 15.12.2018 um 17:12 schrieb Florian Klämpfl:
> The memory manager itself pools already, so no need for the compiler. If 
> somebody wants to improve the heap manager:
> implement OS supported re-allocations (OS can move memory by just shuffling 
> pages).

Very much agree, it's not a user program's job to work around the standard
memory manager in daily use. Doing that is a C++-ism that shouldn't exist in a
sane environment ;-)

I just tested something, and I'm a surprised by how big the difference is. This
simple test is 1.5 times slower in FPC/trunk/win32 than Delphi 2007 and 2.8
times slower for instances of TComponent. Medium-size GetMemory (I tested 123
bytes) is 22 times slower in FPC.
Looks like there is quite some potential there.


const COUNT=1;
var
  t1, t2: dword;
  objs: array[0..1] of TObject;
  i, j: integer;
begin
  t1:= Gettickcount;
  for i := 0 to COUNT - 1 do begin
for j := 0 to high(objs) do
  objs[j]:= TObject.Create;
for j := 0 to high(objs) do
  objs[j].Free;
  end;
  t2:= Gettickcount;

  writeln((t2-t1)/COUNT:10:3, 'ms');
  Readln;
end.

-- 
Regards,
Martok


___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] x86_64 Optimizer Overhaul

[Martok]

Am 12.12.2018 um 04:51 schrieb Ryan Joseph:
> I’ve spent some time in the compiler sources now and I’m curious just where 
> people think the bottle necks for performance actually are. It’s such a 
> complicated system for anyone one person to have a good understanding of so 
> it’s not clear where to begin looking.

Checking out the memory manager(s) could be useful as well - there are a lot of
small allocations, that generally tends to put much stress on it.
And any improvement there would also directly benefit user applications.

-- 
Regards,
Martok


___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] Case code pattern

[Martok]

Am 14.08.2018 um 11:27 schrieb Marco Borsari via fpc-devel:
>  From what I can read from Wikipedia, every compound of the case is 
> enclosed in
> a procedure (or in a function, as you said the table is typed), declared 
> with
> nostackframe, and called by an array of index, right?

The blocks are just regular labels. What I meant was the jump array index is 
considered typed, which allows some optimizations.
This is noteworthy because FPC is the only compiler I could find that does that.


Roughly:

case x of
  0: code;
  1: Morecode;
  2: EvenMoreCode;
  //... assume we have enough case labels to get a jumptable
end;


Gets translated as if one had written:

label label0,label1,label2,{...,}afterend;
const table: array [lowestcaselabel..highestcaselabel] of CodePointer = 
(@label0, @label1, @label2{,...});
if (xhighestcaselabel) then
  goto @afterend;
goto table[x];
label0:
  code;
  goto afterend;
label1:
  Morecode;
  goto afterend;
label2:
  EvenMoreCode;
{...}
afterend:


Iff it follows from the strong typing of the array index that the "if" at the 
start is always false for all defined values of
the type of x, it is omitted.

-- 
Regards,
Martok

Ceterum censeo b32079 esse sanandam.

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] Case code pattern

[Martok]

Hi,

> I would need a clarification about the way the case statement is 
> translated into assembler by FPC. When the list of alternatives is 
> continous, does the compiler generate a jump table?
What Kit said, but a correction: the threshold is not 50, it is 19. And what is 
generated is not technically a jump table, but a
typed dispatch table.


-- 
Regards,
Martok

Ceterum censeo b32079 esse sanandam.

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] Why/how does the compiler have a non-trivial number ofmemory leaks after over two decades of development?

[Martok]

Am 30.07.2018 um 14:24 schrieb Marcos Douglas B. Santos:
> Is performance more important than being correct?  :|
In this project, the answer is always taken to be yes.

-- 
Regards,
Martok


___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] Compile time functions

[Martok]

Am 22.07.2018 um 06:03 schrieb J. Gareth Moreton:
> Pure functions cannot be
> evaluated at compile time if one of the arguments is a variable (or at the 
> very
> least, not deterministic)
Do you have an idea how to prove that yet? The way I see it now, this is the
only hard issue, every other component is already present somewhere in some 
form.

> although there
> might still be issues in exceptional conditions (e.g. division by zero or 
> trying
> to calculate tan(pi/2), for example).
Good point - what should happen in that case?
const
  x  = Pi / 0;
begin
  writeln(x);
end.
That writes "+Inf".

> The other point with a "pure" modifier is that, assuming the compiler 
> determines
> that the function is indeed pure, it would permit their assignment to a 
> constant.
So: on processing a call to a pure function,
  if arguments are constant, emit (interpreted) result of function
  otherwise, emit regular call
?
What would error/warning cases be? I'd want to avoid a situation like that with
the insightful, but unfixable and thus spammy "06058_N_Call to subroutine "$1"
marked as inline is not inlined" message.

> The final issue is that while a function might be pure, you might not want to
> inline it because of its complexity, especially if it's called multiple times
> with variable arguments.
That is very true. Should the "interpretation complexity" be limited, or should
the compiler keep going and maybe run into a stack overflow?


-- 
Regards,
Martok


___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] Compile time functions

[Martok]

Am 21.07.2018 um 02:08 schrieb Sven Barth via fpc-devel:
> The main problem is that not all functions that would be eligible for your
> approach are also declared as inline thus their node trees would not be stored
> inside the PPU and thus you could not work with them.
I'm not sure I understand. Isn't the same true, with a new feature? The full
node tree must be available, which basically means "has inlineinfo". Doesn't it?

> Additional benefit of the "pure" modifier: the compiler can check once the
> method has been parsed whether it's pure or not and can thus error out in the
> latter case.
That is true, it is a more explicit declaration of intent.
Maybe the codegen of pure can simply be implemented as generating inline info,
but always replacing the calls with the simplified version. If it is already
known that a function is pure, what I did with constexpr() would basically be
guaranteed to work.

-- 
Regards,
Martok

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

[fpc-devel] Compile time functions

[Martok]

Hi all,

inspired by Kit's concept for pure functions, I had a deeper look at what inline
functions (and more importantly, the nodes involved in expanding them) can do.
Let me share my proof-of-concept (which may very well become a
'poof-of-concept'...). You can find my current working branch here
<https://github.com/martok/freepascal/compare/master...dev_inlining>.

Instead of marking a function as 'pure' at declaration, I took another way: make
normal inlining firstpass flexible enough that any "accidentally" pure function
can be collapsed at the call site.
As an extension, I then propose a new intrinsic:

function ConstExpr(X: AnyType): AnyType;
If the expression X can be reduced to a constant, the node simplifies to
that constant.
Otherwise, fail compilation with E3203 Illegal Expression.

ConstExpr is basically pass-through for most expressions, but it allows us to
"call" functions /at parse time/, as long as these functions are pure and the
arguments are constant at the point of declaration.

Let me illustrate with an example:

program tinline3;

  function sinc(x: Real): Real; inline;
  begin
    if X = 0 then
  Result:= 1
    else
  Result:= sin(x) / x;
  end;

const
  s1 = constexpr(sinc(0.3));
  s2 = constexpr(sinc(0));
var
  u, v: Real;
begin
  u:= s1;
  u:= s2;
end.

This gets processed into the node tree (right at parse time):


***
after parsing
$main; Register;

***
(blockn, resultdef = $void = "untyped", pos = (16,1), loc = LOC_INVALID,
expectloc = LOC_INVALID, flags = [], cmplx = 4
   (statementn, resultdef = , pos = (17,9), loc = LOC_INVALID,
expectloc = LOC_INVALID, flags = [], cmplx = 4
  (assignn, resultdef = $void = "untyped", pos = (17,3), loc =
LOC_INVALID, expectloc = LOC_INVALID, flags = [], cmplx = 2
 (loadn, resultdef = Real = "Double", pos = (17,3), loc =
LOC_INVALID, expectloc = LOC_INVALID, flags = [nf_write], cmplx = 1
    nil
    symbol = U
 )
 (realconstn, resultdef = Real = "Double", pos = (17,7), loc =
LOC_INVALID, expectloc = LOC_INVALID, flags = [], cmplx = 2
    value =  9.85067355537798561294E-0001
 )
  )

   )
   (statementn, resultdef = , pos = (18,9), loc = LOC_INVALID,
expectloc = LOC_INVALID, flags = [], cmplx = 2
  (assignn, resultdef = $void = "untyped", pos = (18,3), loc =
LOC_INVALID, expectloc = LOC_INVALID, flags = [], cmplx = 2
 (loadn, resultdef = Real = "Double", pos = (18,3), loc =
LOC_INVALID, expectloc = LOC_INVALID, flags = [nf_write], cmplx = 1
    nil
    symbol = U
 )
 (realconstn, resultdef = Real = "Double", pos = (18,7), loc =
LOC_INVALID, expectloc = LOC_INVALID, flags = [], cmplx = 2
    value =  1.E+
 )
  )

   )
)

As you can see, the expressions have been simplified into (the correct)
constants of type Double (Real for x86 is Double). Intermediate type conversions
(including potential loss of FP-precision) are handled correctly (there's a
sneaky int(0)<>real in the example).
This is already really useful for the most common application of macros with
parameters in C: calculating constants (like #define _IOC(inout,group,num,len)
(inout | ((len & IOCPARM_MASK) << 16) | ((group) << 8) | (num))).

There are currently a few issues:

 1. I extended the const propagation optimisation pass a bit. It had
restrictions on inlinen and realconstn that probably had a reason, but I
can't see it...
 2. calln now does an automatic internal optconstpropagate pass when inlining.
This is to carry refs to constant temps a bit further down. That might in
fact be problematic because it could increase code size with multiple loads
and should probably not be done in general. But for this PoC, that was the
simplest way.
 3. procs can only be inlined if their body has been fully parsed before the
call. This is actually kind of a problem because it limits usefulness for
the C-style constant-macro application.
 4. The tricky part (same goes for pure functions) is actually finding out if
parameters of an inlined function are constants. optconstpropagate can track
that after the fact, but at that point the fun is already over. The best
solution I could think of is running some very primitive DFA along while
parsing, so that the firstpass can refer to that. Basically, if a symbol is
known to be constant, loads of it could be treated as constants of its
stored 

Re: [fpc-devel] Progress of pure function research

[Martok]

Am 16.07.2018 um 18:54 schrieb J. Gareth Moreton:
> In that situation, yes it is.  Max is a very simple function though, and it 
> gets
> more complicated if it appears in another pure function where the parameters 
> are
> variables, but whose values are deterministic.
Actually, this might be more workable than I thought at first. The "Thing from
-O3" that carries out the simplification in my example (it also works for mixed
things like "x:= 1; z:= Max(x, 2);") is exactly the constant propagation
optimization. It might be enough to simply force a few localized optimizations
after inlining and before simplifying.
I'll have a look into that later - this would be useful for many cases
regardless of pure functions.

-- 
Regards,
Martok


___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] Progress of pure function research

[Martok]

Am 16.07.2018 um 07:01 schrieb J. Gareth Moreton:
> As stated in the Wiki page, my first test case is the Max function.  Since it
> works both as an inline and a pure function, I can easily change the 
> directive to
> analyse the code flow in the compiler.
I may have missed this in the discussion before. But isn't that a prime example
for "simple" const propagation?

==
function Max(a, b: integer): integer; inline;
begin
  if a > b then
Result:= a
  else
Result:= b;
end;

  z:= Max(1, 2);
==
That already gets optimized to `z:= 2;` on -O1, while the following needs -O2,
but gets to the same result:
==
  x:= 1;
  y:= 2;
  z:= Max(x, y);
==

Tail recursion expansion could do the same for certain recursive functions.


-- 
Regards,
Martok


___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] Issue #32913

[Martok]

Am 14.07.2018 um 20:41 schrieb Dmitry Boyarintsev:
> Exiting from finally? Is it something new?0
> 
> http://docs.embarcadero.com/products/rad_studio/delphiAndcpp2009/HelpUpdate2/EN/html/devcommon/cm_cant_leave_finally_xml.html
Still documented for Tokyo:
<http://docwiki.embarcadero.com/RADStudio/Tokyo/en/E2126_Cannot_BREAK,_CONTINUE_or_EXIT_out_of_a_FINALLY_clause_%28Delphi%29>

Although the example is of course wrong, the way it's given would be 'E2097
BREAK or CONTINUE outside of loop'. Just substitute "break" with "exit".

Control flow statements are allowed in an EXCEPT block. Come to think of it,
didn't we have another TRY/FINALLY/EXCEPT nesting issue that might have
something to do with that?

-- 
Regards,
Martok

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] Managed Types, Undefined Bhaviour

[Martok]

Am 29.06.2018 um 21:27 schrieb Thorsten Engler:
>>> [...] COM interface methods can't logically not be virtual,
>>
>> I think you are confusing things here. They can be virtual or not
>> virtual in COM and CORBA.
> 
> I think a lot of people simply don't understand how interfaces are 
> implemented.
Thank you for the explanation! Saved for future reference.

I was thinking too much in terms of C++ pure virtual classes and their VMT and
forgot about the self translation trampoline functions.


-- 
Regards,
Martok


___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] Managed Types, Undefined Bhaviour

[Martok]

Am 29.06.2018 um 16:37 schrieb Thorsten Engler:
> The specific functions that implement an interface get baked into the class 
> at the moment when the interface is defined as part of the class. This 
> results in important differences in behaviour, depending if methods (in the 
> class) are defined as virtual or not, and if a derived class redeclares an 
> interface already declared on an ancestor or not.
Okay, then why does the calling convention change matters so much?

Maybe a COM/CORBA thing? COM interface methods can't logically not be virtual,
and the kind of code from my example has always worked (for me!) in FPC.

I am confused. Which sorta ties in to the whole "surprises" thing from before we
hijacked this thread...

-- 
Regards,
Martok

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] Managed Types, Undefined Bhaviour

[Martok]

Am 29.06.2018 um 16:05 schrieb Michael Van Canneyt:
>> The expected output would be 3 Addrefs and 3 Releases.
> 
> I don't get that.

Somewhat current FPC trunk output, annotations added manually:
==
Addref: 0022FAA8 Refcount: 1 at 00404961
   (by fpc_class_as_intf in GetChainer)
Addref: 0022FAA8 Refcount: 2 at 00404223
   (by fpc_intf_assign of GetChainer Result)
Release: 0022FAA8 Refcount: 2 at 004041F4
   (by fpc_intf_decr_ref of GetChainer Result)
Chain: 0022FAA8
Addref: 0022FAA8 Refcount: 2 at 00404961
   (by fpc_class_as_intf in Chain)
Addref: 0022FAA8 Refcount: 3 at 00404223
   (by fpc_intf_assign of Chain Result)
Release: 0022FAA8 Refcount: 3 at 004041F4
   (by fpc_intf_decr_ref of Chain Result)
Chain: 0022FAA8
Addref: 0022FAA8 Refcount: 3 at 00404961
Addref: 0022FAA8 Refcount: 4 at 00404223
Release: 0022FAA8 Refcount: 4 at 004041F4
Done: 0022FAA8
fin
Release: 0022FAA8 Refcount: 3 at 004041F4
   (by fpc_intf_decr_ref at scope end of Test)
Release: 0022FAA8 Refcount: 2 at 004041F4
   (dito)
Release: 0022FAA8 Refcount: 1 at 004041F4
   (dito)
==


Delphi output (without the stack trace part, because they don't have it):
==
Addref: 0205DBE8 Refcount: 1
Chain: 0205DBE8
Addref: 0205DBE8 Refcount: 2
Chain: 0205DBE8
Addref: 0205DBE8 Refcount: 3
Done: 0205DBE8
fin
Release: 0205DBE8 Refcount: 3
Release: 0205DBE8 Refcount: 2
Release: 0205DBE8 Refcount: 1
==


Delphi uses a shortcut for "as", and because of their different handling of
putting Result in a tempvar, they get away with less internal assignments.

6 LOCK instructions (and associated calls) less. Not a lot by itself, but since
we're counting single-digit cycles in other places...


-- 
Regards,
Martok


___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] Managed Types, Undefined Bhaviour

[Martok]

Am 29.06.2018 um 14:41 schrieb Michael Van Canneyt:
> As far as I can see, you get 2 chain and 1 done call. Which is what I'd 
> expect.
> The overrides of the _* calls are useless, since they are not virtual in
> TInterfacedObject and hence never called. So that's OK too.
Interface functions are always virtual and implemented by the actually
instantiated class. The "override" keyword is neither allowed nor needed, this
code does what it looks like.

The expected output would be 3 Addrefs and 3 Releases.
A bit better would be doing the last release after "Done" and before "fin" (but
this is difficult because of the implied exception frame - there are cases where
Delphi does it anyway, depending on method length).
The "ideal" output would get away with even less (but I don't think this is
possible without translating to SSA form first and doing some strict counting).

The observed output is 6 Addrefs and 6 Releases. At some point in the past (this
may have to do with variable and register allocation), a Release could happen
too soon. It doesn't now, so that's good.

There is nothing technically wrong with the generated code, but it is not nearly
as efficient as it could be. See also Ryan's comments about slow
Interlocked*-calls a few weeks ago. Delphi's output for the same example is
better, giving the expected output.
Because of the tempvars, it is also not exactly what one might expect at first,
which is why I mentioned it in this context.

-- 
Regards,
Martok


___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] Managed Types, Undefined Bhaviour

[Martok]

Am 29.06.2018 um 12:43 schrieb Michael Van Canneyt:
> Out of curiosity, can you give a simple example of such a funny behaviour
> in such a chaining pattern ?
We've had this topic about 2 years ago with regard to automatic file close on
interface release. Interestingly, something must have changed in the mean time,
because the trivial testcase is now *different* , which is somewhat the point of
being weird-undefined ;-)

Take this example: https://pastebin.com/gsdVXWAi

The tempvar used to get reused, causing lifetime issues with the "chain object".
This isn't the case anymore, now three independent tempvars are used, all of
which live until the end of the function, potentially keeping the object alive
for a long time.
There is also one fpc_intf_assign with associated addref/release per as
operator, which isn't technically necessary.

One could probably avoid the interfaces here with ARC records, but either I'm
missing something or the scope lifetime of tempvars there is even worse.

-- 
Regards,
Martok


___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] Managed Types, Undefined Bhaviour

[Martok]

> I hope this issue gets addressed, as I deem the current behaviour completely
> broken and also going totally against the spirit of Pascal, feeling much more
> like some very obscure behaviour I'd expect from some C compiler.
> Discovering the handling of this issue, however, makes me wonder
> whether fpc aims to be a great Pascal compiler that does without bad surprises
> and very very hard to debug "documented" behaviour or not.
There is less undefined behaviour than in C, but the one we have will bite you
in the most awful ways, sometimes after years of working just fine. And we don't
even have a nice formal standards document that one could grep for "undefined".

But yeah, as Jonas wrote, this _isn't_ one of these occasions. FPC uses (and
reuses!) tempvars a lot more than Delphi, which causes all sorts of funny
behaviours with managed types. Try returning a string or use the
JavaScript-style "Foo().Bar().Baz()" method chaining pattern and you'll see what
I mean.

Change your function to the following, and it will do what one would expect:
function DoSomething (len: longint): Vector;
begin
  Result:= nil;
  SetLength (result,len); // whatever
end;

For managed types, as far as I can tell:
 - locals are initialized (even if there is a warning telling you they are not)
 - tempvars are initialized *once*
 - Result is never initialized (there is no warning telling you it is not).


-- 
Regards,
Martok

Ceterum censeo b32079 esse sanandam.

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] Possible internal corruption

[Martok]

> A clue that leads me to believe there's internal corruption is that a produced
> .s file yields an alignment field of ".balign 119,0x90", which should never
> happen.
Could you set a breakpoint on aggas.pas:721 (the call to doalign) with a
conditional on "tai_align_abstract(hp).aligntype=119" and check what the actual
type of hp is? It could be that at some point a node gets its typ changed in an
invalid way?
aligntype should be either one of 2^[0..5], never something else...

This is where AddressSanitizer support would be *nice*.


-- 
Regards,
Martok

Ceterum censeo b32079 esse sanandam.

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] Debugging Loop Unroll Optimization

[Martok]

Am 21.05.2018 um 19:27 schrieb Ondrej Pokorny:
> Well there is still something left:
You are right, variables don't correctly get captured (or rather, don't get
captured at all). This is not good.
Dirty fix: only unroll loops over local variables when there are no nested
procs. But that probably prohibits most useful cases as well...

I'd say that closures + AST-level inlining + some dead store eliminations would
fix a lot of issues that currently have special case handling.

-- 
Regards,
Martok


___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] Debugging Loop Unroll Optimization

[Martok]

Am 21.05.2018 um 17:44 schrieb Florian Klämpfl:
> I added raise, exit, goto and label as well.
Oh, label, right.

I'd say #0033614 can be resolved as "fixed in 39083" and #0033753 as "no change
required" then.

-- 
Regards,
Martok


___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] Debugging Loop Unroll Optimization

[Martok]

Am 18.05.2018 um 17:15 schrieb Sven Barth via fpc-devel:
> Maybe it should also check for goto and at least explicit raise statements? 
That'd be adding goton and raisen to the check, right?

For now, just checking exit seems to be enough to get rid all of the easily
testable Lazarus crashes. The "Optimized IDE" profile is usable again. I'll post
the patch to the bug tracker, as we seem to have decided it is not a hack ;-)

-- 
Regards,
Martok


___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] Debugging Loop Unroll Optimization

[Martok]

> Sorry to waste your time on this.
Don't worry, I like investigating this stuff. I don't like the rule-lawyering
that usually follows ;-)

>  I'm glad it states the for-loop variable will be left undefined - that's 
> good enough documentation for me.
It is *not* undefined when the loop is left with break or exit, that's the
point. The ISO is not a very good reference for modern-ish Pascal.

-- 
Regards,
Martok

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] Debugging Loop Unroll Optimization

[Martok]

> Citation: "If the loop was terminated prematurely with an exception or a 
> break statement, the loop variable retains the value it had when the 
> loop was exited."
As a quick fix, not unrolling loops left with exit at least fixes this specific
situation. This still leaves exceptions raised, but IIRC the handlers don't
restore context anyways, we might be okay?

diff --git a/compiler/optloop.pas b/compiler/optloop.pas
index 46039ffc5a..dc714ea2cc 100644
--- a/compiler/optloop.pas
+++ b/compiler/optloop.pas
@@ -76,7 +76,7 @@ unit optloop;

 function checkbreakcontinue(var n:tnode; arg: pointer): foreachnoderesult;
   begin
-if n.nodetype in [breakn,continuen] then
+if n.nodetype in [breakn,continuen,exitn] then
   result:=fen_norecurse_true
 else
   result:=fen_false;

I'll be running this on today's snapshot, see if anything else remains.

-- 
Regards,
Martok


___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] Debugging Loop Unroll Optimization

[Martok]

Am 18.05.2018 um 14:07 schrieb J. Gareth Moreton:
> What's the easiest solution to this? Change the code or change the rule?
Follow consensus. There's little point in ignoring 30 years of language
development, just because nobody started an ISO committee.

In this case: """After a for-statement is executed, other than being left by a
goto-statement, the control-variable shall be undefined"""
Break/continue/exit wasn't even invented then!

Borland extended this, hence the VCL samples mentioned.


But that is a discussion I have wasted a lot of time on once already, no
intention to do that again.

-- 
Regards,
Martok


___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] Debugging Loop Unroll Optimization

[Martok]

Am 17.05.2018 um 21:06 schrieb Florian Klämpfl:
> -O3 probably "breaks" a lot of code which exploits the limits of a language-
True.

I do wonder if it would be good to create some sort of collection of things that
are undefined in the core language and how different compilers and dialects
handle them. It's been 28 years, there's probably consensus on some out there.
Could be useful as a guide for porting code as well.


Martok

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] Debugging Loop Unroll Optimization

[Martok]

> Can you report that to the bugtracker of Lazarus?
Sure. Done as https://bugs.freepascal.org/view.php?id=33753

Martok

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] Debugging Loop Unroll Optimization

[Martok]

Same documentation for FPC, 
<https://www.freepascal.org/docs-html/ref/refsu58.html>

But, someone clearly explicitly thought otherwise at some point:
<https://svn.freepascal.org/cgi-bin/viewvc.cgi/trunk/compiler/aasmcnst.pas?view=markup#l860>
That warning is only "wrong" if Result is well-defined after exiting the loop.

> So this shall not be unrolled, but is still error prone, if Result is not set 
> after the regular end
> of the for loop.
Am I doing something wrong, or does FPC not emit a warning when a loop variable
is read after the loop? Delphi has a warning, and does indeed display it for the
test case. There is no indication anything might be wrong from FPC?

This kind of code is used even more than "that other thing", makes me wonder if
it's a good idea to break this at O3...

-- 
Regards,
Martok


___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] Debugging Loop Unroll Optimization

[Martok]

I think I found something, see below. Answering the question first ;-)

Am 16.05.2018 um 19:20 schrieb Florian Klämpfl:
> How big is the project? Normally, the number of unrolled loops is reasonable 
> small so comparing the
> generated assembler with and without unrolling should be feasible.According 
> to cloc, 40kLOC, 10kLOC of which are DSP code, not counting packages
that are not in the source repo. With the very spurious crashes, not the best
test case...

I did, however, do the insane thing and diff Lazarus (or rather, IDAs output).
There are a lot of unrolled loops in the LCL alone, but the code reordering for
the first 15% or so that I checked looks fine. {Off-Topic: could parts of that
variable rewriting be used for more aggressive inlining?}


Some more testing with Lazarus and -gt has shown that at some variables are
uninitialized with -Ooloopunroll. There is a fairly common case where removing
the variable is not good:
-
for Result:= alow to ahigh do
  if Something(Result) then exit;
-
If that gets unrolled, Result is undefined, and whatever it gets assigned to
receives the random content of r/eax.

I'm 90% sure that global variables and the Result variable are *not* supposed to
be undefined after the loop (in contrast to Delphi-compatible locals), but can't
seem to find a reference for that. This pattern is used all over the place, so
somebody else must have thought so too? Grepping for "for result:=" over the FPC
source tree gives 10 matches, Lazarus has over 100 results.

Test case:
-
{$Optimization LOOPUNROLL}

var
  i : integer;
  s : single;

function tfun: integer;
begin
  for Result:=1 to 10 do
s:=s+1;
end;

begin
  s:=0.0;
  for i:=1 to 2 do
s:=s+1;
  if i <> 2 then
halt(1);
  i:= tfun();
  if i <> 10 then
halt(2);
  if s <> 12 then
halt(3);
  writeln('ok');
end.
-

I would very much expect that to be the main cause of the observed crashes.

--
Regards,
Martok


___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

[fpc-devel] Debugging Loop Unroll Optimization

[Martok]

Hi all,

as we have discovered in 0033576 and 0033614, there is a bug somewhere in the
loop unroll optimization pass that only appears in complex enough code. The
problem is, it doesn't happen in the single testsuite test, and the observable
crashes in Lazarus are caused by memory corruption at some point unrelated to
the crash, so it's hard to debug (at least on Windows, without rr...).

I have one other project that has sporadic crashes with -OoLOOPUNROLL (I wish I
had figured that out back then), but that is about as difficult as Lazarus,
where it's at least 100% reproducible.

Does anyone have more complete test cases, or maybe smaller affected projects?

-- 
Regards,
Martok

Ceterum censeo b32079 esse sanandam.

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] Dangerous optimization in CASE..OF

[Martok]

Am 21.04.2018 um 16:40 schrieb Alexander Grotewohl:
> To be honest I agree with what he's saying. We're bending enums to do 
> things normal people just wouldn't (and shouldn't) do.
No, we don't.

There are two semantically different sorts of enums: if you want an enum to
contain only the named elements and nothing else, just use a simple enum without
assigned values. That tells the compiler that you don't really care about
numerical values (except that they are ordinal), the named atoms matter to you.
That implies that the in-memory storage may be opaque.
This is the sort of Enums languages like Ada or Oberon have, and that we inherit
from ISO and TP modes (although they were explicitly documented in TP as
internally being the second type!).

Using assigned values tells the compiler that you *do* care about the ordinal
values. This means that we now need a specific host-type behind it (one that
contains at least all the integers you assigned). Delphi and FPC use a subrange
of the type indicated by $PACKENUM (like a minimum envelope), C-Style languages
use one of the basic integer types as that host-type (because they don't have
subranges). In any case, all values of the host-type are now valid values of the
enum, not just those that have names.
You might think of it like that:
type
  TProgrammerType = (tpJava=2, tpVisualC=4, tpDelphi=100,
 tpClang=101, tpVB=255);
could be roughly rewritten as:
  TProgrammerType = record
  public type
OrdinalType = 2..255;   // sizeof(OrdinalType) := $PACKENUM
  public const
tpJava= OrdinalType(2);
tpVisualC = OrdinalType(4);
tpDelphi  = OrdinalType(100);
tpClang   = OrdinalType(101);
tpVB  = OrdinalType(255);
  end;
You'll notice that if you compile that code normally,
sizeof(TProgrammerType.OrdinalType) would be 1. The storage requirement is
enforced separately from the formal type system, which is essentially the issue
at the very root of this rather long and windy thread.
This would look a bit more obvious in C#, where you might write:
  enum ProgrammerType : ushort = {tpJava=2, tpVisualC=4, tpDelphi=100,
 tpClang=101, tpVB=255};
This syntax makes it very clear that ProgrammerType is a redefinition of ushort,
with some typed constants. The literal translation to plug in to the code above
could be
  public type
OrdinalType = type Word;
Otherwise this is the same.


I hope this clears things up a bit.

-- 
Regards,
Martok

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] Dangerous optimization in CASE..OF

[Martok]

Am 20.04.2018 um 21:35 schrieb Ondrej Pokorny:
> Sven (or anybody else), could you please comment on
> https://bugs.freepascal.org/view.php?id=33603 ? I feel I am getting crazy.
That bug became a textbook example of Cipolla's fourth law, sorry.


> From what I read, there seems to be a difference between FPC and Delphi
> understanding of "enums with holes":
Not even that, the two documentations are not exclusive. Nothing about assigned
enumerated types as documented by FPC excludes Delphi's rules. Which is why I
never understood that:

> Also, the difference is demonstrated in the fact that Delphi and FPC delphi 
> mode allow to define an array of Size. OBJFPC mode doesn't allow it.
It may have something to do with typeinfo (see #27622).
For fpc_read_text_enum et al., a different table is generated, which is correct
even for assigned enumerated types.

> An enumerated type is, in effect, a subrange whose lowest and highest values
> correspond to the lowest and highest ordinalities of the constants in the
> declaration. [...] Only three of these values have names, but the
> others are accessible through typecasts and through routines such as Pred, 
> Succ,
> Inc, and Dec.
The same is true in TP, and by implementation in FPC.

> Therefore I enabled the IS/AS operators on enums with holes only in Delphi 
> mode
> and disabled them in all other modes.
Please do not do this. The other modes are the only ones that really need it, in
Delphi mode, nothing should break on invalid enum values anyway (if it does, it
is a compiler error).


-- 
Regards,
Martok

Ceterum censeo b32079 esse sanandam.

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] Type Compatibility: dynarray <=> Pointer

[Martok]

Am 17.04.2018 um 11:52 schrieb Stefan Glienke:
> FWIW this has been changed in Delphi 10.2 and does not compile anymore but 
> gives:
> E2010 Incompatible types: 'Dynamic array' and 'Pointer'
That is what I would have expected.
Could you please test what the following does in D10.2?

var arr: array of byte;
begin
  arr:= Pointer(0);
end.

What I'd like to know: do they check the "nil" literal, or pointers of value 
nil?

-- 
Regards,
Martok


___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] Type Compatibility: dynarray <=> Pointer

[Martok]

Am 17.04.2018 um 01:51 schrieb Thorsten Engler:
> And the nil assignment variant is pretty much ubiquitous in any code 
> involving dynamic arrays that I'm aware of. 
Yes. I know ;-)

>> only the reserved/constant "nil" is compatible, and handled elsewhere
I asked specifically about assigning arbitrary untyped pointers.


___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

[fpc-devel] Type Compatibility: dynarray <=> Pointer

[Martok]

Hi all,

I have started debugging 0031215, and discovered something slightly unrelated,
but odd. I hope someone can clear that up.
While testing, I found out that the following compiles, and becomes a call to
fpc_dynarray_assign:

var
  arr: array of byte;
begin
  arr:= Pointer(42);
end.

This eventually happens because of the conversion case arraydef<=pointerdef in
defcmp.pas:1151, where a comment says: "nil and voidpointers are compatible with
dyn. arrays". This comment was there since the very first SVN import.

My question: why *are* voidpointers assignment compatible to dynarrays? It does
seem to be Delphi compatible, but I couldn't find any mention in either
documentation that this is possible - only the reserved/constant "nil" is
compatible, and handled elsewhere.

-- 
Regards,
Martok


___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] Dangerous optimization in CASE..OF

[Martok]

Am 15.04.2018 um 11:09 schrieb Ondrej Pokorny:
> Please note that I want to support all ordinal values on the left side 
> of the operator. You can then validate a value of variable of the enum 
> type itself.
Useful indeed! It still strikes me as weird that one will need to add this check
many times when porting code from Delphi, but at least now one can.

Jonas will probably argue that this is a tautology and should always evaluate to
true because a variable of a type by definition is of that type, regardless of
content, but it might be considered as somewhat in line with how "var is class"
handles nil values as "not an instance of class".

-- 
Regards,
Martok

Ceterum censeo b32079 esse sanandam.

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] Dangerous optimization in CASE..OF

[Martok]

Regarding things to be documented in Delphi: As was established last year, this
aspect of FPCs type system is intended to be a bit more strict than that of the
Hejlsberg languages (*), so I'm not taking anything for granted any more. (Case
in point, while I see it: the very different role of range checks on subranges.
The "Percentile" example a bit down from your link is completely invalid in FPCs
type system, but valid *and* failing rangechecks ever since TP5).

(*) Referring to TP, Delphi, and C# - which all share the same characteristics.

Am 13.04.2018 um 21:21 schrieb Ondrej Pokorny:
> Why? I don't think so. Enums with assigned values are documented to be valid 
> in
> the whole range Low..High:
> 
> http://docwiki.embarcadero.com/RADStudio/Tokyo/en/Simple_Types_(Delphi)#Enumerated_Types_with_Explicitly_Assigned_Ordinality
Most obvious issue: do for loops iterate over the holes in enums?

But I wasn't just referring to enums with assigned values. Even simple enums are
not as simple, as soon as they formally don't fall back on their host type.

type
 enu = (aa, bb, cc, dd);
var
  v: enu;
begin
  v:= dd;   // doesn't matter how dd ends up in v
  v:= succ(v);  // always invalid, no need for {$R+}!
end.

All this is not quite as easy to get right as it seems on the surface. But I do
like the the "v is TEnum" operator from the other side of this thread,
regardless of where this goes...

-- 
Regards,
Martok

Ceterum censeo b32079 esse sanandam.

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] Dangerous optimization in CASE..OF

[Martok]

Am 13.04.2018 um 12:52 schrieb Ondrej Pokorny:
> I introduced the AS operator for enumerators in
> https://bugs.freepascal.org/view.php?id=33603
I'm still not convinced that cementing the FPC-ism of Ada-style high-level enums
is a good idea (and how that is even logically supposed to work with assigned
values), but if we want to go there, something like this feature is absolutely
required (Ada has it).

In that case, off the top of my head, succ/pred, for, bitsizeof and maybe sizeof
need to be fixed as well.

-- 
Regards,
Martok

Ceterum censeo b32079 esse sanandam.

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] Wrong docs: not initialized global variables

[Martok]

Am 05.04.2018 um 08:35 schrieb Michael Van Canneyt:
> If the compiler devs wanted, they could initialize every string with the
> '' constant,
That is in fact the -gt option.

> Pascal states: do not assume that variables are initialized.
Corollary: there is no guarantee that "class operator Initialize()" is ever
called, and Maciej can roll back management operators. I very much doubt this is
what you want to imply.
To preempt the argument - no, this is not a different case. Managed types are
initialized at the latest when they come into scope, period.

> From this rule, it follows that every variable must be explicitly initialized 
> [...]
> Be it with an assignment or an 'var a: type = someonstant;'.
... for which the syntactic sugar was rejected not two weeks ago.



To the people referring to Borland Language Guides: As I painfully discovered,
the Borland guides are *not* considered normative material for FPC by certain
FPC developers. Not even for -Mtp.
For some reason, ISO 7185 *is* a reference, even for decidedly not-ISO modes.

Am 05.04.2018 um 12:47 schrieb Alexander Klenin:
> Allow me to yet again to single out this philosophy of
> strongly preferring abstract purity to concrete user experience.
If it at least was consistent purity!

Sorry. Needed to be said.


-- 
Regards,
Martok

Ceterum censeo b32079 esse sanandam.

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] Redundant compiler hints

[Martok]

The hint was added in response to 
<https://bugs.freepascal.org/view.php?id=31717>.

Not very useful, other than to demonstrate 'inline' is rarely respected.
However, it probably wasn't the intention to generate it for intrinsics as well?

-- 
Regards,
Martok

Ceterum censeo b32079 esse sanandam.

Am 25.03.2018 um 11:56 schrieb Ondrej Pokorny:
> Yesterday I updated FPC to current trunk (r38623) and now I get a lot of 
> redundant compiler hints. E.g.:
> 
> program Test;
> uses SysUtils;
> var
>    FS: TFormatSettings;
> begin
>    FS := DefaultFormatSettings;
> end.
> 
> Test.lpr(6,3) Hint: Call to subroutine "procedure 
> $fpc_copy_proc(Src:Pointer;Dest:Pointer;TypeInfo:Pointer);" marked as 
> inline is not inlined
> 
> Anybody knows what's going on? I am on Windows, use the 32bit compiler.
> 
> Ondrej
> 
> ___
> fpc-devel maillist  -  fpc-devel@lists.freepascal.org
> http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
> 



___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] End of support for Win XP?

[Martok]

Am 01.02.2018 um 14:08 schrieb Denis Kozlov:
> It still feels *very early* to drop support for Windows XP. I haven't used it
> properly in years, but I can't say the same about the target user audience. I
> still test some builds against Windows XP.

This may be a very stupid question, but please believe me it is not as troll-y
as it might sound...

What would be the point of supporting older CPUs, if we don't actually support
the OS that runs/ran on them? Or the other way around, there'd be no point in
generating code compatible with a specific processor if the minimum supported
*OS* doesn't run on that cpu.
I'm specifically not talking about the compiler itself (or Lazarus), just being
able to target a platform. That mostly limits the RTL I guess.

I still support Win2000 with one application (with just a few compat shims), and
used to have a Delphi 4 install specifically for one (industry) machine running
NT4 at a previous gig. At my current one, we have a Win95 box running expensive
hardware. True, I ported that program to Win32 last year (Did you know BC++ 5
runs on Win10? I didn't.), but still... that stuff is not as rare as one might 
hope.

What I'm trying to say: it would be amazing if there was a way to be
forward-compatible without entirely scrapping old stuff. Maybe do the same as MS
does, and have IFDEFs for the API level in the headers? I wouldn't mind having
to recompile the RTL.


By the way: Dropping XP support also drops ReactOS support. I'm not sure if we
ever officially had it, but it is on the Wiki.


Martok

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] Question about $OPTIMIZATION LEVELn

[Martok]

Am 17.01.2018 um 16:38 schrieb Jonas Maebe:
> The LevelX switches are only intended for internal use by the compiler, 
> and were never intended to be exposed.
Good, that's what I thought too.

> The ability to have the equivalent of 
> -O1/-O2/-O3/-O4 in source code was never planned.
Florian wrote in 0025873,
> But there is indeed no equivalent for -O3/4 at source level ({$optimizations 
> on} equals to -O2). I'am open for suggestions how such a directive should 
> look like. 

May I propose to redefine {$optimization levelN} to do that? Reasons:
a) we agree the flags alone (should) have no use
b) it's always been documented that way
c) it's used as if that was true in live code

Rough patch attached...

-- 
Regards,
Martok

Ceterum censeo b32079 esse sanandam.
Index: scandir.pas
===
--- scandir.pas (revision 37943)
+++ scandir.pas (working copy)
@@ -1039,17 +1039,20 @@
 current_scanner.skipspace;
 { Support also the ON and OFF as switch }
 hs:=current_scanner.readid;
-if (hs='ON') then
-  current_settings.optimizerswitches:=level2optimizerswitches
-else if (hs='OFF') then
-  current_settings.optimizerswitches:=[]
-else if (hs='DEFAULT') then
-  current_settings.optimizerswitches:=init_settings.optimizerswitches
+case hs of
+  'OFF': current_settings.optimizerswitches:=[];
+  'ON': current_settings.optimizerswitches:=level2optimizerswitches;
+  'DEFAULT': 
current_settings.optimizerswitches:=init_settings.optimizerswitches;
+  'LEVEL1': 
current_settings.optimizerswitches:=level1optimizerswitches;
+  'LEVEL2': 
current_settings.optimizerswitches:=level2optimizerswitches;
+  'LEVEL3': 
current_settings.optimizerswitches:=level3optimizerswitches;
+  'LEVEL4': 
current_settings.optimizerswitches:=level4optimizerswitches;
 else
   begin
 if not UpdateOptimizerStr(hs,current_settings.optimizerswitches) 
then
   Message1(scan_e_illegal_optimization_specifier,hs);
   end;
+end;
   end;
 
 procedure dir_overflowchecks;
___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] Question about $OPTIMIZATION LEVELn

[Martok]

> No - see the discussion at https://bugs.freepascal.org/view.php?id=25873
Huh.

Quick googling shows lots of projects out there that clearly assumed it would.
So we need to get at least the documentation fixed for the future.

If I were to replace {$OPTIMISATIONS ON} with {$OPTIMISATIONS LEVEL4} somewhere,
that switches most optimizations *off* instead of doing more. Not terribly
intuitive.

On the other hand, there is no way to say "do all medium-expensive optimizations
you know on this code": listing them individually is cumbersome, not portable
and not extensible for when new flags are introduced.

Is there even a use case for the current behaviour? I.e., when would one
actually mean -Oolevel3 instead of -O3?


-- 
Regards,
Martok

Ceterum censeo b32079 esse sanandam.

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

[fpc-devel] Question about $OPTIMIZATION LEVELn

[Martok]

Hi all,

just a quick question: is setting one of the LEVELn values like
  {$Optimization LEVEL3}
in a source file supposed to be functionally equivalent to calling the compiler
for that file with -O3?

If so, then there's a bug: only the level gets applied, none of the associated
switches (i.e. level3optimizerswitches, including the inherited lower levels)
are set. This seems weird?
Patch seems straightforward, I'd just want some input on what the negative
options like {$Optimization NOLEVEL3} should do.

Otherwise it's a documentation bug:
<https://www.freepascal.org/docs-html/prog/progsu58.html> claims {$OPTIMIZATION
ON} is equivalent to {$OPTIMIZATION LEVEL2}, which it never was since that
directive was introduced.


Slightly related, the Programmer's Guide lists ancient optimization switches in
11.3. Bug report coming in shortly.

-- 
Regards,
Martok

Ceterum censeo b32079 esse sanandam.

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] FillWord, FillDWord and FillQWord are very poorly optimised on Win64 (not sure about x86-64 on Linux)

[Martok]

Am 01.11.2017 um 05:58 schrieb J. Gareth Moreton:
> So I've been doing some playing around recently, and noticed that while 
> FillChar has some very fast internal 
> code for initialising a block of memory, making use of non-temporal hints and 
> memory fences, the versions 
> for the larger types fall back to slow Pascal code.
It might be worth it to look at the Pascal versions from generic.inc first, and
see if it is possible to come up with versions that generate faster code.

I'm actually surprised "REP STOSD" should be that much faster. I remember it
being slower on modern platforms than it used to be?

-- 
Regards,
Martok

Ceterum censeo b32079 esse sanandam.

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] x86_64.inc CompareByte

[Martok]

Using the code given below as "inner", I measure this:

Current Trunk:
O0 compare-byte-1 : 196065.112 +/- 896.754 cycles/inner [0.5 %CV 1.6 %R]
O1 compare-byte-1 : 196510.158 +/- 577.976 cycles/inner [0.3 %CV 1.1 %R]
O3 compare-byte-1 : 187540.922 +/- 706.167 cycles/inner [0.4 %CV 1.5 %R]
Patch from 2017-10-21:
O0 compare-byte-2 : 175831.632 +/- 965.972 cycles/inner [0.5 %CV 2.1 %R]
O1 compare-byte-2 : 176039.560 +/- 527.141 cycles/inner [0.3 %CV 1.0 %R]
O3 compare-byte-2 : 158527.167 +/- 661.690 cycles/inner [0.4 %CV 1.5 %R]
(%CV: coefficient of variance * 100%. %R: span as % of mean)

CPU:
 Intel(R) Core(TM) i5-4200M CPU @ 2.50GHz Family 6 Model 60 Stepping 3 (Haswell)
 true single core clock (measured) 2.83 GHz


So the new version is a bit faster, but not by a large margin (10-15%). It is
statistically significant though.
While I'm at it, i386 could use some love:
O1 compare-byte-1 :  755247.183 +/- 8125.671 cycles/inner [1.1 %CV 4.5 %R]
That's 3.8 times slower than x64 for exactly the same code.

Code:
len:=random(100);
for j:=0 to len-1 do
  begin
buf1[j]:=random(256);
buf2[j]:=random(256);
  end;

for j:=0 to random(10) do
  buf2[j]:=buf1[j];

for j:=1 to 1 do
  CompareBytePatch(buf1,buf2,len);  // or System.CompareByte


-- 
Regards,
Martok

Ceterum censeo b32079 esse sanandam.

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] rdtscp

[Martok]

Hi,

Am 21.10.2017 um 21:41 schrieb Wolf:
> rdtsc cannot do it either. You need to have a CPU capable of 
> understanding rdtscp.
From what I understood, that doesn't give you cycles either, but only the same
timestamp intervals RDTSC returns.
Tangent: On Windows, RDTSC is wrapped by the QueryPerformanceCounter() call. QPC
incidentally is complicated enough that it is very likely no out-of-order
instructions are pending by the time it gets to actually executing RDTSC, but
with less jitter than "abusing" CPUID for serialisation (QPC takes a rather
stable 22 cycles). /Tangent

What I do to get cycle-accurate counts in microbenchmarks is first calibrate the
timer with a known-cycle-length task, obtain a "timestamps per cylce" from that
(depending on core clock, usually between 200 and 1300), and then measure the
actual function. Note that depending on how well you can control multitasking,
interrupts and power management, you will need thousands to millions of repeats
of the function under test to be reasonably free from artefacts.
Looks a bit statistical, but it's precise enough to actually see the instruction
cache, instruction alignment and branch prediction at work. I can also validate
Agner Fog's Instruction Timing Tables with it, so it can't be that bad ;-)

-- 
Regards,
Martok

Ceterum censeo b32079 esse sanandam.

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] Dangerous optimization in CASE..OF

[Martok]

Hi all,

another few months, and something funny happened this morning: a tweet shows up
in my timeline which links to this article on efficient codegen for dispatch
with switch statements:
<http://www.cipht.net/2017/10/03/are-jump-tables-always-fastest.html>
Very interesting!

Why I'm resurrecting this thread is the reference to Arthur Sale's 1981 paper
"The Implementation of Case Statements in Pascal". He compares linear lists,
jumptables, binary search and masksearch (on B6700 machines). The bit on
jumptables (journal-page 933) contains this part:
"""The jump-table itself consists of half-word (3 bytes) unconditional branches,
and must be half-word synchronized. The range-check and indexed branch add up to
23 bytes of instructions on the assumption that the range limit values are
fitted into 8-bit literals, and there is a 0-2 byte padding required to achieve
jump-table synchronization. *The range check is never omitted as the
consequences of a wild branch which lands outside the jump-table are potentially
disastrous.* If the range is r, the space requirements are therefore [...]"""

"potentially disastrous" probably didn't mean security as much as "my room-sized
mainframe crashes", but the point stands...

-- 
Martok

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

[fpc-devel] Use of Enumerations in header translations

[Martok]

Hi all,

so, some time ago, Jonas asked this:

> I don't know, but I still don't understand why on Earth you would want
> them [binary-compatible C-style enums] in a strongly typed language.
To which my reply was: header translations!

I've spent some time on Saturday with ppudump and jq to find out what
enumerations with offsets are used for in the FPC tree itself. (Mainly because I
wanted to know what in your opinion the intended and correct application is - I
still *really* want to fix that and get back to it every couple of days.)

The results are interesting: use in packages is exclusively header translations
and on-disk file formats.
Use in the compiler itself is widespread too: the TRegister range type is an
enum where the whole point is that it can contain any value of the base type.
tmsgstate is an enum (with jumps). Binary constants for DWARF are enums.
tinlinenumber is an enum.

All of which I have been told in no uncertain terms are not intended to be safe
and should just not be done.

Many people clearly thought otherwise at some point.

A really good and short header example of the clean and very Pascal-y code
possible by translating enums as enums is the NVAPI header. One instance of our
original issue I could quickly Google up is something like this:
--
var bt: NV_GPU_BUS_TYPE;
NvAPI_GPU_GetBusType(hPhysicalGPU, bt);
case bt of
  NVAPI_GPU_BUS_TYPE_UNDEFINED   : WriteLn('Driver doesn''t know hardware!');
  NVAPI_GPU_BUS_TYPE_PCI : WriteLn('You have a PCI card!');
  NVAPI_GPU_BUS_TYPE_AGP : WriteLn('You have an AGP card!');
  NVAPI_GPU_BUS_TYPE_PCI_EXPRESS : WriteLn('You have a PCIe card!');
  NVAPI_GPU_BUS_TYPE_FPCI: WriteLn('You have an FPCI card!');
else
  WriteLn('You have some future interface!');
end;
--
Looks perfectly sane, right?
NV have in the meantime extended the upstream C enum to include
NVAPI_GPU_BUS_TYPE_AXI = 5. We have an else statement to catch that, but the
compiler may choose to not even generate code for it. This is bad.

Of course, AndiH wrote that header first for Delphi, where it is reliably 
safe...


Why am I posting this? Well, if it is really your intention that enums are only
safe to read if all writes happen in Pascal code without casts, then all of
those imports are plain and simple wrong, and must be changed. If enumerated
types can only formally contain the values explicitly named, then the entire
codegen is undefined.
Or we could just agree that C-style enums must be treated as low-level, which
some committers obviously assumed anyway.


Affected packages visible from win32 target:
==
compiler itself (see type names above)
rtl (windows.pp)

cairo
fcl-db
fcl-sdo
fcl-xml
fftw(very minor)
gtk1(with more affected
gtk2 code in the LCL)
httpd22
httpd24
libcurl
libenet
libusb
libvlc
libxml2
mad
mysql
nvapi
oracle
winunits-base
winunits-jedi
zorba


--
Martok


___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] Dangerous optimization in CASE..OF

[Martok]

Am 25.07.2017 um 19:31 schrieb Martok:
> As has just been pointed out to me, we all misdiagnosed that example.
Turns out this is not a new question, there is actually a very thorough
treatment of that very issue on SO:
<https://stackoverflow.com/questions/18195312/what-happens-if-you-static-cast-invalid-value-to-enum-class>

(Continue reading after the mention of CWG 1766, the interesting part is what
"representable range" means)

The simple answer however is: a C++ enum with fixed base type (FPC analogue: any
setting other than {$PACKENUM DEFAULT}), any value of the base type is valid.
C++ enums *without* base type may also have a smaller range than int, but always
a power-of-two. That would mean one could mask with that size (we already
generate the masking sometimes) and make the jumptable larger, saving a Jx at
the expense of a larger table.

So the question boils down to: do we want C-Style Enums to behave like in
C-dialects, or just look like they do?

If we do, there's a very simple solution: by setting a fixed $Z option, the
programmer specifically *chose* Low-Level enums. We could just honour that and
be done with it.

--
Martok

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] Dangerous optimization in CASE..OF

[Martok]

As has just been pointed out to me, we all misdiagnosed that example.

TL;DR: you did test undefined behaviour, only a different one. The example
actually proves that clang and gcc agree with the MSDN article in that (even
simple) C++ enums are low-level.


I have verified that Clang/LLVM generates the same code as GCC for the switch
statement itself. It does correctly check for the maximum jmptable index (via
[sub 0x7;ja]) and only then jumps.

LLVM however does generate an UD2(0F0B) trap instruction for several programmer
errors, such as the control flow reaching the end of a non-void function without
return. *That* is why we get a SIGILL when no switch label matches.
In -O1 and -Os, these debug instructions are removed again.

As predicted, there does indeed appear to be a Clang bug: GCC correctly warns
"control reaches end of non-void function", while Clang only emits the UD2
instruction (so it detected it) and does not print the warning.

Completing the function works as expected:

> *snip*
>   return 2;
> case e8:
>   printf("Hello\n");
>   return 3;
>   }
return -1;
> }
> 
> int main()
> *snip*

Another equivalent solution is to have the return in the switch statement's
default label. No UD2 is emitted then.
It follows that the compiler concludes that the default can be matched, even
when all named elements are listed. Therefore, enum variables may contain
unnamed values. Therefore, C++ enums must be Low-Level. QED.


> "Ungültiger Maschinenbefehl (Speicherabzug geschrieben)" = Invalid opcode 
> (memory dump written).
> Why? Because it does not range check before entering the jump table.
I really should have noticed that. A jump into nonexecutable memory would be
SIGSEGV, not SIGILL.

--
Martok

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] Dangerous optimization in CASE..OF

[Martok]

> OK, I see now: there is a difference between C enums and C++ enums. Your 
> example was about C++ enums. My example was about C enums. The C enums 
> are defined to allow any integer value, whereas C++ enums are strongly 
> typed.
In the pages cited, there's no mention of valid ranges, only that for C++ enums,
you need static_cast<>, and that "In the original C and C++ enum types, the
unqualified enumerators are visible throughout the scope in which the enum is
declared. In scoped enums, the enumerator name must be qualified by the enum
type name." That's just our $SCOPEDENUMS switch.
So it really is undefined and clang takes the unsafe option. Sounds familiar.


> C enums: https://msdn.microsoft.com/en-us/library/whbyts4t.aspx 
> https://www.gnu.org/software/gnu-c-manual/gnu-c-manual.pdf
> C++ enums: https://msdn.microsoft.com/en-us/library/2dzy4k6e.aspx
C# enums:
https://docs.microsoft.com/en-us/dotnet/csharp/language-reference/keywords/enum

"A variable of type Days can be assigned any value in the range of the
underlying type; the values are not limited to the named constants."

I mentioned that in passing before, if we take reference from a C-style
language, we should probably use one that shares more ideas (and the lead 
designer).
___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] Dangerous optimization in CASE..OF

[Martok]

Am 16.07.2017 um 13:17 schrieb Jonas Maebe:
> Does that mean that you would consider the same transformation of a 
> case-statement when using a subrange type as correct? And that putting a 
> value outside the range of a subrange into such a variable as a 
> programmer error? (as opposed to doing the same with a non-subrange enum 
> type?)
Hold on, there already is a test for that particular question in the language
itself!

 -> Can the type be used as an array index?

---
{$mode objfpc}
type
  TExplEnum = (a=1, b=3, c=5, d=7);
  TEnArr = array[TExplEnum] of Byte;
---
=> Error: enums with assignments cannot be used as array index

Makes sense, after all, what should happen with the gaps? And creating the array
for the entire base type with lots of filler data would potentially be too
memory-consuming.

However:
---
{$mode objfpc}
type
  TExplEnum = (a=1, b=3, c=5, d=7);
  TSubEnum = a..d;
  TEnArr = array[TSubEnum] of Byte;

begin
  WriteLn('SizeOf(TEnArr) = ', SizeOf(TEnArr));
  WriteLn('Low(TEnArr) = ', Low(TEnArr), ', ', Ord(Low(TEnArr)));
  WriteLn('High(TEnArr) = ', High(TEnArr), ', ', Ord(High(TEnArr)));
end.
---
SizeOf(TEnArr) = 7
Low(TEnArr) = a, 1
High(TEnArr) = d, 7
---

That difference was unexpected. At least for me.


In {$mode delphi} (and Delphi), we get the second result for both tests. So
there already is some distinction of enum semantics between modes.



Also:
---
  k:= Pred(c);
---
Error: succ or pred on enums with assignments not possible

---
  k:= Pred(TSubEnum(c));
---
Happily compiles.


So, from the compiler's perspective, we cannot rely on the values of enums with
assignments enough to use them as an index (or count them), but we can do so
with subranges - because the subrange in question is effectively 1..7 and
doesn't actually know (or care) about the enum-ness of its host type.


Huh. Fascinating.
___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] Dangerous optimization in CASE..OF

[Martok]

You (Florian) do realize that it's almost impossible to write a C++ program that
is not technically undefined? Their 'standards' are worse than our
'implementation-defined'.

FWIW, GCC agrees with Low-Level Enums, and given that clang regularly catches
hate when their 'optimizations' break stuff like the Linux kernel again...

g++:
  40058b:   8b 45 fcmov-0x4(%rbp),%eax
  40058e:   83 f8 07cmp$0x7,%eax
  400591:   77 76   ja 400609 <_Z1f5tenum+0x89>
  400593:   89 c0   mov%eax,%eax
  400595:   48 8b 04 c5 d8 06 40mov0x4006d8(,%rax,8),%rax
  40059c:   00
  40059d:   ff e0   jmpq   *%rax

g++ -O3:
  4005a4:   83 ff 07cmp$0x7,%edi
  4005a7:   77 14   ja 4005bd <_Z1f5tenum+0x1d>
  4005a9:   89 ff   mov%edi,%edi
  4005ab:   ff 24 fd 18 07 40 00jmpq   *0x400718(,%rdi,8)

Proving my point that we should aim to be better and safer than C, not worse.
___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] Dangerous optimization in CASE..OF

[Martok]

Am 16.07.2017 um 19:58 schrieb Ondrej Pokorny:
> On 16.07.2017 19:24, Martok wrote:
>> The good thing about case statements is that they tell me of every other
>> programmer error: missing elements (if used without else)
> Off-topic: how can I enable this compiler hint?
Erm, I was referring to the "normal" DFA, ie. for function results or variable
initialization.

type
  TEnum = (one, two);

function GetInteger(A: TEnum): Integer;
begin
  case A of
one: Result:= 1;
  end;
end;

... which for some reason only Warns in -O3, and then it's "wrong" sometimes
too, because DFA assumes that enums are Low-Level enums. That was the other
thread on this list recently.

Yeah. Probably a bad argument, sorry.

Martok
___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] Dangerous optimization in CASE..OF

[Martok]

Am 16.07.2017 um 16:34 schrieb DaWorm:
> Does the compiler optimize away the else clause in this case?  Seems to me it
> should not.  At least that isn't the behavior I would expect as a user.
At least it should not do that without telling me. The good thing about case
statements is that they tell me of every other programmer error: missing
elements (if used without else), elements that aren't type-correct, double
elements... but not 'extra' else-blocks.


Am 16.07.2017 um 18:26 schrieb DaWorm:
> Academically that may be true, but in the real world that code wouldn't be
> unreachable.  I write code that deals with communication protocols all the
> time.  I can't control what the other side sends.  I have two choices. Write a
> lot of code to validate each and every element is within the proper range, or
> let the handler for each element, that I have to write anyway, deal with the
> unexpected values.
> It can be worked around by casting all parts of the case to integer, but that
> leads to ugly code.
That is exactly my discovery use case, and is also why I keep calling this a
remote code execution: it breaks on sensible network-facing code in a scary way.

Example code snippet from libOpenPGP:
<https://pastebin.com/2wsfCXfP>
One of the more obvious places, this pattern repeats all over the project, from
data parsing all the way down to simple enum-to-string for logging
(SignatureTypeToStr, PKAlgorithmToStr).

It's just a coincidence this is currently partly safe (none of the *ToStr
functions are!), if I was to add a label with value pkPrivate110 (because I
already have pkRSA) and implement the elliptic curve signatures, we would be
jumping to attacker-controlled memory locations. We currently do that in all of
the *ToStr-functions, instead of executing the else-blocks.

That code is completely unambiguous and well-defined if we assume Low-Level
Enumerations (which, coming from BP, I obviously always have).


Martok
___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] Dangerous optimization in CASE..OF

[Martok]

> And you also have subranges of enum types. Can any assumptions made 
> about those in your opinion?

> Does that mean that you would consider the same transformation of a
> case-statement when using a subrange type as correct? And that putting a
> value outside the range of a subrange into such a variable as a
> programmer error? (as opposed to doing the same with a non-subrange enum
> type?)
Depends on the compiler version sadly :/

Subranges for TP5 are documented as "don't rely on anything at runtime, we only
check compiletime", TP7 documents "outside range is an RTE (independent of $R
state)", Delphi is documented like TP5 again.

My intuition was shaped by learning the language with D4 (and D3 books), but
I've always thought that as weird and makes subranges a bit pointless.

I would think that
type
  TEnum = (a,b,c);
  TSubEnum = a..c;

should have the same semantics, but at the same time they can't if subranges are
strict and enums are not. I see now where you're coming from.
(I'll get back to that example at the end.)

And then there's bitpacked records...


> But I finally understand where the disconnect comes from. I have always 
> thought of enums as more or less equivalents of subrange types, simply 
> with an optional name for the values. You, and indeed the Pascal 
> standards, treat them differently.
Getting back to the terms Ondrej introduced yesterday, I think that "normal"
enums may or may not be High-Level enumerations, but enums with explicit
assigment can *only* be Low-Level enumerations. Can we safely distinguish them
in the compiler? Does it even make sense to add that complexity?

This gets weirder. I think Borland already made that distinction, but... not?
<http://docwiki.embarcadero.com/RADStudio/XE5/en/Simple_Types#Enumerated_Types_with_Explicitly_Assigned_Ordinality>

"""An enumerated type is, in effect, a subrange whose lowest and highest values
correspond to the lowest and highest ordinalities of the constants in the
declaration. [...] but the others are accessible through typecasts and through
routines such as Pred, Succ, Inc, and Dec."""

So that's about the "gaps": they're valid, just unnamed.
But for subranges, they write:

"""incrementing or decrementing past the boundary of a subrange simply converts
the value to the base type."""
So we can also leave the min..max range and transparently drop to the parent
type. This raises in $R+, _but is valid otherwise_. (* This is the exact same
text as in the TP5 langref *)

Logical conclusion from that: a variable of a subrange of a
 1) High-Level enum becomes invalid when we leave the declared enum elements
 2) Low-Level enum remains valid by way of dropping to the base type.
Having both variants in the type system is too complex IMO - although it would
be something where the programmer clearly has to state her intentions.


My initial proposed trivial solution was to keep this undefined (maybe document
the difference to BP), and simply change codegen to be undefined-safe normally
and only undefined-unsafe in -O4. I am, however, no longer so sure if that is
really a good solution.

There has to be a reason why everybody else chose Low-Level enums, except that
it is far simpler to implement, right?


> And it would also require us to conditionalise every future optimisation 
> based on type, in particular separating the treatment of enums from that 
> of integers. That's a lot of (future) work and care to deal with what I 
> still consider to be bad programming.
Delphi optimizes always based on the full-range base type:

type
 TB = (a,b,c,d,e); // Sizeof(TB)=1
 TT = a..e;
var
  t: TT;
begin
  t:= TT(2);
  if t <= e then   // does not get removed
  if Ord(t) <= 255 then// 'Condition is always true'




Martok
___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] Dangerous optimization in CASE..OF

[Martok]

> This will never generate a range check error, because the type 
> information states that a tsubenum2 value is always a valid tsubenum 
> value. Array indexing a special case of this, as semantically the 
> expression you use to index the array is first assigned to the range 
> type of the array.
> 
> I would assume that this is something that "someone with a solid 
> knowledge of the language" would expect.
Probably. Subranges are after all explicit subsets of something. But let's not
digress, right? That's not related to the topic at hand.


> but plain comparisons are removed at compile-time:*With* a warning. Two, 
> actually.

>> However, FPC does not have the luxury of being the first to define and 
>> implement
>> a new language (well, except for $mode FPC and ObjFPC). There is precedent.
> 
> At least the precedent in ISO Pascal 
> (http://www.standardpascal.org/iso7185rules.html) is that you cannot 
> convert anything else to an enum, and hence an enum by design always 
> contains a value that is valid for that type (unless you did not 
> initialise it all, in which case the result is obviously undefined as well).
I know this website, turns out that's not quite what ISO7185 says. The ISO is
awfully unspecific about what you can or cannot do with enums. They simply
define enumerated types as defining a set of constants with values 0,1,2 etc.,
and later the compatibility-rules you cite below.

But even if we take the web version:
"""Enumerated types are fundamentally different from integer and subrange types
in the fact that they cannot be freely converted to and from each other."""

'fundamentally different from [...] subrange types' - what I said above.

'cannot be freely converted to and from *each other*' - what they mean by that
is that

type y = (red, green, blue);
type day = (mon, tue, wed, thur, fri, sat, sun);
var
  color: y;
begin
  color:= fri;
end.

will not work. I don't think anyone would want that ;-)

In any case, we have mode ISO for being extra-ISO-compatible - there are some
significant differences between Borland Pascal and ISO/IEC already. Probably
that mode should also receive the "non-bindable" limitation you cite from
IEC10206. I just noticed: case..else should be a syntax error there,
it doesn't exist in ISO7185 and should be case..otherwise in IEC10206 - where it
is technically mandatory, because a non-matching argument is a dynamic-violation
(RTE).

We also have modes TP and Delphi, and at least there it is *not* an error to
have an unnamed value in a variable, because (spoken in terms of the ISO) the
ordinal-type of an enumerated-type *is* the base type, not a (potentially
non-consecutive) subrange. I've quoted the relevant parts of the language
references multiple times already.
Low/High (and the compiler-internal analogue of them - cf. function getrange()
in FPC) produce the first/last element, but that's it - for example Pred/Succ
may produce unnamed elements.

type
 TT = (a=2,b,c=7,d,e);  // defines constants of type TT for 2,3,7,8,9
{$R+}
var
  t: TT;
begin
  t:= b;
  t:= succ(t);
  Writeln(ord(t)); // writes '4'
end.

Note that FPC doesn't accept this code in mode (Obj)FPC, but correctly does so
in DELPHI, with the same result as Delphi.


Added after Ondrej's message 20:52: Borland appears to have taken the route of
what he called a 'LOW-LEVEL enumeration' from the very beginning.


Martok




___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] Dangerous optimization in CASE..OF

[Martok]

Am 15.07.2017 um 12:40 schrieb Jonas Maebe:
> On 14/07/17 02:40, Martok wrote:
>> There is a fundamental difference in the type system between a somewhat
>> sensible (if unexpected) assumption in FPC and a more practical documented
>> definition in every other Pascal compiler. An assumption that even FPC
>> follows only in this one single spot.
> 
> Several times in this thread I've already given examples in this thread that
> this is not true.
And several times in this thread I've shown that the places you mention will
behave the same whether we have strict enums or not - they are correct for
either interpretation, simply by doing what a developer without knowledge of the
specific compiler internals, but with solid knowledge of the language has
come to expect.

For example, if I index an array, I know bad things may happen if I don't check
the index beforehand, so I must always do that.
That if the compiler makes up the array access somewhere along the way sometimes
no check happens is not very predictable.

> and in comparisons that get optimised away at compile time because they will
> always have the same result at run time according to the type information.
I've shown that is not the case for the more obvious expressions in the forum
post linked above.
Several different ways of writing the (apparent) tautology "is EnumVar in
Low(EnumType)..High(EnumType)" all handle out-of-range-values (expressly, not as
a side effect of something else). Which is especially noteworthy because with
strict enums, we might as well drop the elseblock entirely and warn "unreachable
code" in these tests.
> If a data location has a particular type but does not contain a value that is
> valid for that type (e.g. because it has not been initialised with one, or
> because an invalid value was put there via an explicit type cast or assembler
> code), then the result is undefined. Note that "undefined" does not mean "the
> code will crash". It is one possibility, but in the general sense it means
> "anything could happen".
Absolutely true.
However, FPC does not have the luxury of being the first to define and implement
a new language (well, except for $mode FPC and ObjFPC). There is precedent. And
that precedent is Conclusion 1 of my post above: Enums are handled as a
redefinition of the base type with constants for the names. Some intrinsics
(pred/succ) and the use of the type itself (array[TEnumType], set of) use the
enum-ness for something, most don't. There is nothing undefined.
Do not confuse the additional treatment added by {$R+} with the basic defined
behaviour.


Martok
___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] Dangerous optimization in CASE..OF

[Martok]

Am 14.07.2017 um 10:04 schrieb Marco van de Voort:
> In our previous episode, Martok said:
>> There is a fundamental difference in the type system between a somewhat 
>> sensible
>> (if unexpected) assumption in FPC and a more practical documented definition 
>> in
>> every other Pascal compiler. An assumption that even FPC follows only in this
>> one single spot.
>> This is unexpected and breaks unrelated code. That's the problem.
> 
> Other pascal's don't have sparse enums ?

Wait, what do sparse enums have to do with any of that?

But: yes, they do.
___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] Dangerous optimization in CASE..OF

[Martok]

Am 13.07.2017 um 22:24 schrieb Marco van de Voort:
> Personally I think the input validation angle to justify checking enums is
> dragged-by-the-hairs. 
I completely agree with you on that. Although in a different way ;-)

That was just the easily-observable breakage of a common pattern. If anybody
actually read what I wrote after Florian clarified the actual issue, I already
narrowed it down to 'simple' compatibility and self-consistency.

There is a fundamental difference in the type system between a somewhat sensible
(if unexpected) assumption in FPC and a more practical documented definition in
every other Pascal compiler. An assumption that even FPC follows only in this
one single spot.
This is unexpected and breaks unrelated code. That's the problem.


Good night,

Martok

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] Dangerous optimization in CASE..OF

[Martok]

Hi all,

any new ideas on this issue?

I've been thinking about this a lot, and I do see where you're coming from.
There is some theoretical advantage in treating enums like that. Only one minor
issue: a language with that interpretation does not appear to be Pascal...

You can find some results of my investigations here:
<https://www.entwickler-ecke.de/viewtopic.php?p=707764#707764>
(German-language forum post, but I know many of the core team are or can read
German anyway; I can provide a translation if you want)

Regardless of whether there may be some argument for this language change, I'm
still a firm believer in "don't surprise the user". There is literally no
precedent that this simplification has ever been done in any Pascal compiler
(quite the contrary), and there is no written hint that FPC does it either.
Basically, if people with some 30-ish years of experience (and always keeping up
with current language extensions) write that, I think we may have an issue here:

> In TP und {$R+} würde aValue ausserhalb einen RangeCheckError erzeugen.
> 
> In {$R-} nicht, jedenfalls solange der Datentyp nicht überfahren wird {$Z..}.
> 
> Demnach sollte also der Sprung in den else-Zweig immer eindeutig definiert 
> sein.
> 
> Jede andere Reaktion würde ich für ein Sicherheitsproblem halten, da hätte 
> Pascal ja keinen Vorteil mehr.


I also read all of ncg*.pas again with respect to range simplifications, and it
turns out that there really is only one instance where we simplify to undefined
behaviour: tcgcasenode. tcginnode just produces the else-branch faster for
x>=high(setbasetype) (without bittests), but is still defined. All others work
with the base integer type only.
Point is: there is really no unrelated side effect at all if we were to align
FPC with all the other Pascals out there.


Kind regards,

Martok



___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] Dangerous optimization in CASE..OF

[Martok]

Hi all,

Am 02.07.2017 um 22:02 schrieb Florian Klämpfl:
> Am 02.07.2017 um 21:40 schrieb Martok:
>> Honestly, I still don't understand why we're even having this discussion.
> Because it is a fundamental question: if there is any defined behavior 
> possible if a variable
> contains an invalid value. I consider a value outside of the declared range 
> as invalid
So, as this is the core of all this, I have spent the last few days asking
various users of pascal languages in different compilers, intentionally without
telling them what this was about. Not a single one considered out-of-range
ordinal values as something bad (though not terribly useful), especially not
causing undefined behaviour: all assumed that they would continue to behave like
ordinals in comparisons.

Something I hadn't known, and which I find quite funny: that group apparently
includes Anders Hejlsberg, who wrote the original Turbo Pascal compiler and
years later specifically defined C# enums contrary to your assumption. In fact,
this entire thread's topic is an actual example in the language reference:
<https://docs.microsoft.com/en-gb/dotnet/csharp/language-reference/keywords/enum>

I haven't yet told all of them why I asked (one set of answers comes from a
forum thread that I don't want to spoil yet, maybe tomorrow evening), but those
who I asked in private all have at some point written code that relies on that
concept and are "irritated" why that wouldn't work in FPC.

All that seems to leave only one conclusion...


Kind regards,

Martok
___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel