Re: [fpc-devel] Overflow in TMemoryStream?

[Martok]

Hi,

yes, I can confirm this as an overflow, but on its own, it should be safe. Above
430MB, the stream doesn't grow by a quarter but just by however much was
requested, luckily the branch fails before the wrong capacity could be set.

Test:
type
  TMS2 = class(TMemoryStream) end;
var
  ms: TMS2;
  ds: Int64;
begin
  ds:= 100*1000*1000;
  ms:= TMS2.Create;
  ms.SetSize(ds);
  WriteLn(ds:15,' ', ms.Size:15, ' ', ms.Capacity:15);
  inc(ds, ds div 10); // grow by less than 25%
  ms.SetSize(ds);
  WriteLn(ds:15,' ', ms.Size:15, ' ', ms.Capacity:15);
end.

with ds=100M, prints:
  1   1   13840
  11000   11000   125005824<< grew by 1/4*100M

with ds=500M, prints:
  5   5   52816
  55000   55000   550002688<< bug, grew by 1/10*500M

However, with ds=869M, prints:
  86900   86900   869003264
  95590 18666185569013440   955904000
and mostly crashes with Runtime Error 203 except when I'm step-by-step-debugging
it...
That looks like a *separate* overflow to me, probably caused by the wild mix of
Int64 and Longint that our Streams inherited from Delphi...

I don't have RTL built with full symbols right now, maybe someone else can
investigate?


Martok

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] Overflow in TMemoryStream?

[Martok]

> I have committed a patch. Please test and report if it is fixed.
> I don't have a 32-bit system available to test on...
Tested on win32: the overflow is fixed, 500M gets incremented by 125M.

I think the RunError is caused by the way ReallocMem works: growing from 869M to
1086M seems to be done by allocating the new area and copying, so there's a
period of almost all of the 2G heap used...


___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

[fpc-devel] Generics: issue with double inline specialization

[Martok]

Hi everyone,

I had already reported the issue as
<http://bugs.freepascal.org/view.php?id=30626>, but as this problem is currently
blocking a clean solution in a project for us, I'm asking for help again here.

The problem appears to be that when a generic uses its type parameter to
inline-specialize another generic (ie. inherit from it), that generic cannot be
specialized anywhere else (ie. return value of that type) or a name collision
occurs. Not really sure why that happens, but for some reason the compiler
doesn't recognize that the two instances don't have the same name by accident
but really are the same type.

Do you have any idea what could be done to work around this?

Thank you in advance,

Martok

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

[fpc-devel] Optimization of redundant mov's

[Martok]

Hi all,

there has been some discussion about FPCs optimizer in #31444, prompting me to
investigate some of my own code. Generally speaking the generated assembler is
not all that bad (I like how it uses LEA for almost all integer arithmetics),
but I keep seeing sections with lots of redundant MOVs.

Example, from a SHA512 implementation:
CurrentHash is a field of the current class, compiled with anything above -O2,
-CpCOREAVX2, -Px86_64.

 a:= CurrentHash[0]; b:= CurrentHash[1]; c:= CurrentHash[2]; d:= CurrentHash[3];
000100074943 488b8424a002 mov0x2a0(%rsp),%rax
00010007494B 4c8b5038 mov0x38(%rax),%r10
00010007494F 488b8424a002 mov0x2a0(%rsp),%rax
000100074957 4c8b5840 mov0x40(%rax),%r11
00010007495B 488b9424a002 mov0x2a0(%rsp),%rdx
000100074963 488b4248 mov0x48(%rdx),%rax
000100074967 488b9424a002 mov0x2a0(%rsp),%rdx
00010007496F 488b6a50 mov0x50(%rdx),%rbp

Every single one of the "mov 0x2a0(%rsp), %rxx" instructions except the first is
redundant and causes another memory round-trip. At the same time, more registers
are used, which probably makes other optimizations more difficult, especially
when something similar happens on i386.

Now, the fun part: I haven't been able to build a simple test that causes the
same issue (the self-pointer already is in %rcx and not fetched from the stack
each time), so I have a feeling this may be a side effect of some other part of
the code.

Does this sound familiar to anyone? If so, what could I do about it?


Regards,

Martok

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] Optimization of redundant mov's

[Martok]

Hi,

> It's called register spilling: once there are no registers left to hold
> values, the compiler has to pick registers whose value will be kept in
> memory instead.
I thought it would be something like that...

Still, my main issue was with the repeated fetches. I'd (naively!) say that it
should be relatively easy for an assembly-level optimizer to detect that these
are repeated loads of the same thing, with nothing that could affect the outcome
inbetween. It's not even a CSE in the technical sense, not a sub-expression but
the entire thing...

> E.g. those memory loads
> are probably optimised by the processor itself (not necessarily coming
> even from the L1 cache, but possibly from the write-back buffer).
Not as well as one might believe, manually fixing (by forcing @CurrentHash into
a register with a local variable) just those 4 lines gives a ~2% increase in
MB/s for this hash. Which is quite a lot, given this is the part *without*
actual computations.

And again, I've seen this happen more than once on i386 code, where it even
creates "fake" register pressure (by using 2 or more registers to hold exactly
the same temporary) that makes the rest of the code worse than it could be.
As a ballpark: the same change as above results in a 10% speedup by freeing up 2
registers (all-int64 operations on i386, so 2 regs needed for everything, having
one more is very noticeable...)

It just strikes me as odd to have some rather good local code but then just
pointlessly add the second-most expensive operation in between ;-)


Regards,

Martok



___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] LineInfo

[Martok]

>> Does it is possible that the LineInfo trace (-gl option) are broken (no 
>> output)
>> in 3.0.2 version on Linux (at least)?
> 
> Hm. Indeed. I can reproduce the issue :/
AFAIR lineinfo.pp only works with Stabs? Didn't the default change to Dwarf?


Martok





___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] UTF-8 string literals

[Martok]

> You should weigh the advantages you outline here against the disadvantages of
> no longer knowing how string literals will be encoded.
As a programmer, either I don't want to know (declared const without giving
explicit type) or I do, then I did declare it correctly:

{$codepage utf8}
var u: UTF8String = 'äöüالعَرَبِيَّة';
  -> UTF8String containing the characters I entered in the source file (in this
case(!!) just 1:1 copy).

{$codepage utf8}
var u: UCS4String= 'äöü';
  -> UCS4 encoded Version, either 00e4 00f6 00fc or the equivalent
with combining characters

There should probably be an error if the characters I typed don't actually exist
in the declared type (emoji in an UCS2String), but otherwise, there's no good
reason why that shouldn't "just work".

> It means e.g. the resource string tables will have entries that are UTF16 
> encoded
> or entries that are UTF8 encoded, depending on the unit they come from. 
> This is highly undesirable.
Always convert from "unit CP" to UTF8 (or UTF16 if some binary compat is
required), done. Aren't they just internal anyway?

> By forcing everything UTF16 we ensure delphi compatibility (yes it does 
> matter) 
> and we also ensure a uniform set of string tables.
If that was what happened, ok. But from the error message Matthias listed as (1)
I would assume that the actual string type is UCS2String, at least at some point
in the process.

Just my 2 cents...

Martok

PS: adding to the discussion over on the Lazarus ML: I just found a fourth wiki
page describing a slightly different Unicode support. This is getting 
ridiculous.


___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] UTF-8 string literals

[Martok]

> That might be the one from Michael Schnell. Probably it should be marked with 
> a
> big, fat warning that it's merely a user's suggestion and nothing official.
Not even that. This one looks relatively obvious to me ;)

I've filed a bug as <https://bugs.freepascal.org/view.php?id=31758> for 
reference.


Martok

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] UTF-8 string literals

[Martok]

That's the one I also think Sven was talking about.
I just searched for "Unicode". Michael's proposal comes up, but I guess the
title is fairly obvious.


But apparently everything is rainbows and unicorns and there is absolutely no
problem with the documentation at all, so I guess this week-long discussion here
never happened anyway.


Martok

Am 10.05.2017 um 08:38 schrieb Mattias Gaertner:
> On Tue, 9 May 2017 14:59:16 +0200
> Michael Schnell  wrote:
> 
>> On 06.05.2017 09:39, Sven Barth via fpc-devel wrote:
>>> That might be the one from Michael Schnell.  
>> Very unlikely, as this text does not mention anything about how a source 
>> file byte sequence is converted in a String constant / literal.
> 
> I think he meant this one:
> http://wiki.lazarus.freepascal.org/index.php?title=not_Delphi_compatible_enhancement_for_Unicode_Support&action=history
> 
> I thought Mschnell is Michael Schnell. Was this wrong?
> 
> Mattias
> ___
> fpc-devel maillist  -  fpc-devel@lists.freepascal.org
> http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
> 


___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] Data flow analysis (dfa) and "case ... of"

[Martok]

Interestingly, I just ran into "bad" code generation with exactly the properties
discussed in this thread.

Take a function like this:

function SignatureSubpacketTypeToStr(const X: TSignatureSubpacketType): String;
begin
  case X of
sstReserved00 : Result:= 'Reserved00';
sstReserved01 : Result:= 'Reserved01';
sstCreationTime   : Result:= 'CreationTime';


Because every declared element is covered, the generated code for it ends up
being a computed goto:

   0x10047c4c <+28>:mov-0x4(%ebp),%al
   0x10047c4f <+31>:and$0xff,%eax
   0x10047c54 <+36>:jmp*0x10071d08(,%eax,4)

Which is perfectly fine if X is guaranteed to be in range of the elements the
case statement matches to. If it is not, as it may be with invalid input data
(as read from a file), that jump goes somewhere undefined - and most
importantly, not into any else statement.

So, while we have code that looks like Result is always properly initialized,
what we get instead is code that doesn't actually work. And no kind of DFA could
detect that, except also range-checking everything.

Just thought I'd share that, as a less synthetic example than some discussed 
here.


Regards,
Martok

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] Data flow analysis (dfa) and "case ... of"

[Martok]

The attitude displayed over on #32079 is, quite frankly, terrifying. Apparently
a language which from the beginning has intrinsics for reading and writing files
must never be used for doing so, or wild things may happen /and that's okay/.

Implying that input should already be sanitized on a bug about something that
breaks input sanitation code (but only sometimes) is just... wow.

If anybody wants it, here's the patch I'll be rolling on the windows snapshots
from now on.


Have a good weekend,
Martok
Index: compiler/ncgset.pas
===
--- compiler/ncgset.pas (revision 36620)
+++ compiler/ncgset.pas (working copy)
@@ -1080,7 +1080,7 @@
labelcnt:=case_count_labels(labels);
{ can we omit the range check of the jump table ? }
getrange(left.resultdef,lv,hv);
-   jumptable_no_range:=(lv=min_label) and (hv=max_label);
+   jumptable_no_range:=(lv=min_label) and (hv=max_label) and 
(cs_opt_level4 in current_settings.optimizerswitches) and not (cs_check_range 
in current_settings.localswitches);
{ hack a little bit, because the range can be greater }
{ than the positive range of a aint}
 
___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

[fpc-devel] Dangerous optimization in CASE..OF

[Martok]

by
'rules-as-written' that is exactly what we should have assumed anyway.

> Just compile with {$RANGECHECKS ON}, then!
I've been trying really hard for the past couple of hours, but I haven't gotten
the compiler to emit a single check at all when doing anything with enums. And
even if, a runtime error is usually not what you want. You'd probably want to
tell the user a file is corrupt instead of killing the program...

> But that still doesn't mean we have to worry about any of this in the
> codegen for CASE..OF - just tell the programmer to manually check their input
> after reading from wherever!
Well, yeah, except... there is no way to do that.
  if EnumValue in [low(TEnumType)..high(TEnumType)] then
will not work for sparse enums or a basetype larger than Byte, and
  case Enumvalue of
All,
Expected,
Values : doSomething;
  else
raise EFileError.Create('Invalid data');
  end;
will obviously also not work because this is just what we're trying to do here
in the first place (NB: this was my original use case).

So, we have a problem here: either the type system is broken because we can put
stuff in a type without being able to check if it actually belongs there, or
Tcgcasenode is broken because it (and _only_ it, as far as I can see) wants to
be clever by omitting an essentially free check for very little benefit.
I know which interpretation I would choose: the one with the easier fix ;-)


I would very much like for someone to at least acknowledge that there is a
problem here, because I can think of several more or less clever fixes for that
(except the obvious) and would prefer discussing these instead of having to
prove that "not-as-defined"-behaviour is not the same as "undefined behaviour".


Kind regards,

Martok
___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] Dangerous optimization in CASE..OF

[Martok]

> Is this made safe by always having an else/otherwise? If so, could the 
> compiler at least raise a warning if an enumeration was sparse but there 
> was no else/otherwise to catch unexpected cases?
Interestingly, not in FPC.

This was also always my intuition that the else block is also triggered for
invalid enum values (the docs even literally say that, "If none of the case
constants match the expression value") - and it *is* true in Delphi. In FPC it
is also mostly true, unless you happen to fall into this optimisation.

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] Dangerous optimization in CASE..OF

[Martok]

Am 02.07.2017 um 10:40 schrieb Michael Van Canneyt:
> These cases are without exception covered by the " unchecked (aka explicit)
> typecast," part of Jonas's statement. Including Read(File).

Aye, that was kinda my point ;)
It is really hard to write code that interacts with the outside world without
having a validation problem.
If the validation code then breaks because the compiler thinks it's clever...

Am 02.07.2017 um 10:29 schrieb Michael Van Canneyt:
> GetEnumName from typinfo will already do this for you.
> We could add an additional function that just returns true or false.
> Something as
> function ValueInEnumRange(TypeInfo : PTypeInfo; AValue : Integer) : boolean;

Enum Typeinfo is horribly broken in so many ways except for the one simple case
needed for published properties, it definitely cannot be used in its current 
form.


That, probably (not sure about the timeline, but it makes sense to me), is part
of the core issue: Enumeration types have become way more powerful since this
optimization was introduced. Back then, nobody would have translated a C library
enum typedef as an enumerated type - simply because we didn't have sparse enums
then. Now we do, and so it is possible to use the typesafe way -- only that it
turns out to be less safe than a byte variable and some untyped constants.


___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] Dangerous optimization in CASE..OF

[Martok]

> Yes, checking the data. I can easily create a similar problem as above with 
> the "range checks" for
> the jump table by reading a negative value into the enum. Unfortunately, the 
> checks are unsigned ...
Actually, fun fact, *fortunately* the checks are unsigned. Having a negative
value (or generally a value before the first element when that has a value
assignment) underflows on the check, and so gets caught by the CMP/JA as well.
Yes, I tried that, your code is safer than you think ;-)

Also, for sparse enums the "gaps" are filled with pointers to else-block, so the
check that is already there turns out to be always safe.

enum = (ela = 5, elb, elc, eld, ele);

1) enum value too small ( = 1):

   mov1,%al
   sub5,%al   # al = -4 = $fb
   cmp(9-5),%al   # $fb > 4
   ja $#ELSE-BLOCK# branches
   and$ff,%eax
   jmp*0x40c000(,%eax,4)

2) enum value in range or in gap (= 7 = elc)

   mov7,%al
   sub5,%al   # al = 2
   cmp(9-5),%al   # 2 <= 4
   ja $#ELSE-BLOCK# no branch
   and$ff,%eax
   jmp*0x40c000(,%eax,4)

3) enum value too large ( = 20)

   mov20,%al
   sub5,%al   # al = 15
   cmp(9-5),%al   # 15 > 4
   ja $#ELSE-BLOCK# branches
   and$ff,%eax
   jmp*0x40c000(,%eax,4)


Same thing on x86_64, where instead of al and eax we use eax and rax, with the
same underflow characteristics.

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] Dangerous optimization in CASE..OF

[Martok]

Addendum to this:

> This was also always my intuition that the else block is also triggered for
> invalid enum values (the docs even literally say that, "If none of the case
> constants match the expression value") - and it *is* true in Delphi.
There is a reason why this is true in Delphi: because this is the way it has
been documented in Borland products for at least 25 years!

I have checked with the TP7 language reference (it pays to keep books around),
which defines the following things:
 - Enumeration element names are implicitly defined as typed constants of their
enum type
 - The enum type is either Byte (<=256 elements) or Word.
 - Subrange types are defined as the smallest type that can contain their range
 - Case statements execute the statements of the matching case label, or the
else block otherwise

Note that they actually defined enumerations as what I called 'fancy constants'
before.


The Delphi 4 language reference (also in book form, which is a bit more detailed
than what is in the .hlp files) uses more precise language:
 - Enumeration element names are implicitly defined as typed constants of their
enum type
 - The enum type is either Byte, Word, or Longword, depending on $Z and element
count
 - Subrange types are defined as the smallest type that can contain their range
 - it is legal to inc/dec outside of a subrange, example from the book:
   type Percentile = 1..99;
   var I: Percentile;
   begin
 I:= 99;
 inc(I);   // I is now 100
   So if this is a legal statement, subrange types can contain values outside of
their range. The description in the German version is "Die Variable wird in
ihren Basistyp umgewandelt", the variable becomes its base type.
 - Case statements execute *precisely one* of their branches: the statements of
the matching case label, or the else block otherwise

So, in D4, we have enums as fancy constants, subrange-types are not safe (so
enums can also never be), and case statements cannot fail.


FPC's language reference has no formal definition of what enums or subranges
really are, and the same language as TP7 regarding case statements.


So at least in modes TP and DELPHI, the optimisation in question is formally 
wrong.

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] Dangerous optimization in CASE..OF

[Martok]

Booleans are not enums in Delphi (not even ordinals), but their own little
thing. "if boolean_expr" is always a jz/jnz, no matter what. They are defined as
0=FALSE and "everything else"=TRUE

However:

var
  b : boolean;
begin
  b:=boolean(3);
  if b = True then
writeln(true)
  else if b = False then
writeln(false)
  else
writeln(ord(b));
end.

That writes 3, which is why your should never compare on the boolean lexicals.
Some Winapi functions returning longbool rely on that.

Wait, that was a trick question, wasn't it?
___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] Dangerous optimization in CASE..OF

[Martok]

> They are:
> http://docwiki.embarcadero.com/Libraries/XE5/en/System.Boolean
That prototype is a recent invention, it wasn't there in older versions. Also
the text sounds quite different somewhere else:
http://docwiki.embarcadero.com/RADStudio/XE5/en/Simple_Types#Boolean_Types

> Yes. What I wanted to point out: also delphi does optimizations on enums 
> which fails if one feeds
> invalid values.
Okay, if you want believe that Booleans are enums:

  b:=boolean(42);
  if not b then
writeln('falsy')
  else
writeln('truthy');

Prints truthy. Doesn't crash.



___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] Dangerous optimization in CASE..OF

[Martok]

Am 02.07.2017 um 20:29 schrieb Ondrej Pokorny:
> On 02.07.2017 20:23, Florian Klämpfl wrote:
>> And the compiler writes no warning during compilation?
> 
> It does indeed.
But about something else.
Can we please stop derailing from the main issue here?


> If we get a convenient way to assign ordinal to enum with range checks, 
> everything will be fine :)
No it will not, we still can no longer elegantly pass/receive enums to/from
libraries from other compilers.
But at least it would be defined then, so programmers would know this is an
incompatibility.


___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] Dangerous optimization in CASE..OF

[Martok]

Am 02.07.2017 um 19:47 schrieb Florian Klämpfl:
> Am 02.07.2017 um 19:29 schrieb Martok:
>>type Percentile = 1..99;
>>var I: Percentile;
>>begin
>>  I:= 99;
>>  inc(I);   // I is now 100
> 
> Forgot the mention:
> Tried with $r+ :)?
That case is also documented. RTE in {$R+}, legal in {$R-}. That also means that
while you could make assumptions about the content in {$R+} (Delphi does not*),
you definitely cannot as soon as there is a single write in {$R-}. A C++
compiler could probably try tracing that using constness of variables and
parameters, but we cannot, and so must be defensive.

*) Even FPC makes no such assumptions in all other instances!

type
  TF = 1..25;
var
  t: TF;
begin
  t:= TF(200);
  if t in [1..50] then  // tautology!
Writeln('a')
  else
writeln('b');

What does that print?
Yeah. As documented.
Check the codegen in R+: the if is still fully generated.
Only tcgcasenode does something else.


Honestly, I still don't understand why we're even having this discussion.
We're not talking about adding a new check - only not leaving one out that is
already there 99% of the time.
We're not talking about standardising some new behaviour - Borland did that
decades ago.
The correct behaviour is already documented in every Pascal language reference
(partly including our own), and is also the intuitive one.

I just don't get it. Why would you sacrifice the runtime safety, or, if you
prefer, the code compatibility, of your compiler over an (arguably wrong in at
least 2 modes) specific technicality of the type system that is adhered to
nowhere else?


Taking a break for now. Grading a thesis starts to sound like good relaxation.

Kind regards,

Martok





___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] Dangerous optimization in CASE..OF

[Martok]

Good morning!

Am 02.07.2017 um 22:02 schrieb Florian Klämpfl:
> Am 02.07.2017 um 21:40 schrieb Martok:
>> Honestly, I still don't understand why we're even having this discussion.
> 
> Because it is a fundamental question: if there is any defined behavior 
> possible if a variable
> contains an invalid value. _I consider a value outside of the declared range 
> as invalid_,
(emphasis mine)
And this is where you disagree with Borland's explicit documentation, Borland's
implicit extensions via consistent compiler behaviour, and with at least ISO
7185:1990 (that revision has no concept of range checks, and explicitly allows
all operations other than constant assignment to exceed a subrange type).
If this is what you always had in mind for the FPC dialect, fair enough. It is
your project, after all :-) I shall submit appropriate change requests for the
documentation, as well as for several other simplifications for all other
conditionals except CASE..OF that then become possible. I will also submit
another set of change requests to *not* do that in modes TP, DELPHI and ISO for
code compatibility reasons. Probably a 'modeswitch strictenums' or something
like that.

To remind you: CASE..OF is currently the only statement that casts this concept
into code (grep -R getrange compiler/*). Everything else is consistent and
compatible.


Regards,

Martok

PS: starting a mail with "good morning" looks rather stupid if one then spends
two hours writing it. Hm.
___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] Dangerous optimization in CASE..OF

[Martok]

Hi all,

Am 02.07.2017 um 22:02 schrieb Florian Klämpfl:
> Am 02.07.2017 um 21:40 schrieb Martok:
>> Honestly, I still don't understand why we're even having this discussion.
> Because it is a fundamental question: if there is any defined behavior 
> possible if a variable
> contains an invalid value. I consider a value outside of the declared range 
> as invalid
So, as this is the core of all this, I have spent the last few days asking
various users of pascal languages in different compilers, intentionally without
telling them what this was about. Not a single one considered out-of-range
ordinal values as something bad (though not terribly useful), especially not
causing undefined behaviour: all assumed that they would continue to behave like
ordinals in comparisons.

Something I hadn't known, and which I find quite funny: that group apparently
includes Anders Hejlsberg, who wrote the original Turbo Pascal compiler and
years later specifically defined C# enums contrary to your assumption. In fact,
this entire thread's topic is an actual example in the language reference:
<https://docs.microsoft.com/en-gb/dotnet/csharp/language-reference/keywords/enum>

I haven't yet told all of them why I asked (one set of answers comes from a
forum thread that I don't want to spoil yet, maybe tomorrow evening), but those
who I asked in private all have at some point written code that relies on that
concept and are "irritated" why that wouldn't work in FPC.

All that seems to leave only one conclusion...


Kind regards,

Martok
___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] Dangerous optimization in CASE..OF

[Martok]

Hi all,

any new ideas on this issue?

I've been thinking about this a lot, and I do see where you're coming from.
There is some theoretical advantage in treating enums like that. Only one minor
issue: a language with that interpretation does not appear to be Pascal...

You can find some results of my investigations here:
<https://www.entwickler-ecke.de/viewtopic.php?p=707764#707764>
(German-language forum post, but I know many of the core team are or can read
German anyway; I can provide a translation if you want)

Regardless of whether there may be some argument for this language change, I'm
still a firm believer in "don't surprise the user". There is literally no
precedent that this simplification has ever been done in any Pascal compiler
(quite the contrary), and there is no written hint that FPC does it either.
Basically, if people with some 30-ish years of experience (and always keeping up
with current language extensions) write that, I think we may have an issue here:

> In TP und {$R+} würde aValue ausserhalb einen RangeCheckError erzeugen.
> 
> In {$R-} nicht, jedenfalls solange der Datentyp nicht überfahren wird {$Z..}.
> 
> Demnach sollte also der Sprung in den else-Zweig immer eindeutig definiert 
> sein.
> 
> Jede andere Reaktion würde ich für ein Sicherheitsproblem halten, da hätte 
> Pascal ja keinen Vorteil mehr.


I also read all of ncg*.pas again with respect to range simplifications, and it
turns out that there really is only one instance where we simplify to undefined
behaviour: tcgcasenode. tcginnode just produces the else-branch faster for
x>=high(setbasetype) (without bittests), but is still defined. All others work
with the base integer type only.
Point is: there is really no unrelated side effect at all if we were to align
FPC with all the other Pascals out there.


Kind regards,

Martok



___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] Dangerous optimization in CASE..OF

[Martok]

Am 13.07.2017 um 22:24 schrieb Marco van de Voort:
> Personally I think the input validation angle to justify checking enums is
> dragged-by-the-hairs. 
I completely agree with you on that. Although in a different way ;-)

That was just the easily-observable breakage of a common pattern. If anybody
actually read what I wrote after Florian clarified the actual issue, I already
narrowed it down to 'simple' compatibility and self-consistency.

There is a fundamental difference in the type system between a somewhat sensible
(if unexpected) assumption in FPC and a more practical documented definition in
every other Pascal compiler. An assumption that even FPC follows only in this
one single spot.
This is unexpected and breaks unrelated code. That's the problem.


Good night,

Martok

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] Dangerous optimization in CASE..OF

[Martok]

Am 14.07.2017 um 10:04 schrieb Marco van de Voort:
> In our previous episode, Martok said:
>> There is a fundamental difference in the type system between a somewhat 
>> sensible
>> (if unexpected) assumption in FPC and a more practical documented definition 
>> in
>> every other Pascal compiler. An assumption that even FPC follows only in this
>> one single spot.
>> This is unexpected and breaks unrelated code. That's the problem.
> 
> Other pascal's don't have sparse enums ?

Wait, what do sparse enums have to do with any of that?

But: yes, they do.
___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] Dangerous optimization in CASE..OF

[Martok]

Am 15.07.2017 um 12:40 schrieb Jonas Maebe:
> On 14/07/17 02:40, Martok wrote:
>> There is a fundamental difference in the type system between a somewhat
>> sensible (if unexpected) assumption in FPC and a more practical documented
>> definition in every other Pascal compiler. An assumption that even FPC
>> follows only in this one single spot.
> 
> Several times in this thread I've already given examples in this thread that
> this is not true.
And several times in this thread I've shown that the places you mention will
behave the same whether we have strict enums or not - they are correct for
either interpretation, simply by doing what a developer without knowledge of the
specific compiler internals, but with solid knowledge of the language has
come to expect.

For example, if I index an array, I know bad things may happen if I don't check
the index beforehand, so I must always do that.
That if the compiler makes up the array access somewhere along the way sometimes
no check happens is not very predictable.

> and in comparisons that get optimised away at compile time because they will
> always have the same result at run time according to the type information.
I've shown that is not the case for the more obvious expressions in the forum
post linked above.
Several different ways of writing the (apparent) tautology "is EnumVar in
Low(EnumType)..High(EnumType)" all handle out-of-range-values (expressly, not as
a side effect of something else). Which is especially noteworthy because with
strict enums, we might as well drop the elseblock entirely and warn "unreachable
code" in these tests.
> If a data location has a particular type but does not contain a value that is
> valid for that type (e.g. because it has not been initialised with one, or
> because an invalid value was put there via an explicit type cast or assembler
> code), then the result is undefined. Note that "undefined" does not mean "the
> code will crash". It is one possibility, but in the general sense it means
> "anything could happen".
Absolutely true.
However, FPC does not have the luxury of being the first to define and implement
a new language (well, except for $mode FPC and ObjFPC). There is precedent. And
that precedent is Conclusion 1 of my post above: Enums are handled as a
redefinition of the base type with constants for the names. Some intrinsics
(pred/succ) and the use of the type itself (array[TEnumType], set of) use the
enum-ness for something, most don't. There is nothing undefined.
Do not confuse the additional treatment added by {$R+} with the basic defined
behaviour.


Martok
___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] Dangerous optimization in CASE..OF

[Martok]

> This will never generate a range check error, because the type 
> information states that a tsubenum2 value is always a valid tsubenum 
> value. Array indexing a special case of this, as semantically the 
> expression you use to index the array is first assigned to the range 
> type of the array.
> 
> I would assume that this is something that "someone with a solid 
> knowledge of the language" would expect.
Probably. Subranges are after all explicit subsets of something. But let's not
digress, right? That's not related to the topic at hand.


> but plain comparisons are removed at compile-time:*With* a warning. Two, 
> actually.

>> However, FPC does not have the luxury of being the first to define and 
>> implement
>> a new language (well, except for $mode FPC and ObjFPC). There is precedent.
> 
> At least the precedent in ISO Pascal 
> (http://www.standardpascal.org/iso7185rules.html) is that you cannot 
> convert anything else to an enum, and hence an enum by design always 
> contains a value that is valid for that type (unless you did not 
> initialise it all, in which case the result is obviously undefined as well).
I know this website, turns out that's not quite what ISO7185 says. The ISO is
awfully unspecific about what you can or cannot do with enums. They simply
define enumerated types as defining a set of constants with values 0,1,2 etc.,
and later the compatibility-rules you cite below.

But even if we take the web version:
"""Enumerated types are fundamentally different from integer and subrange types
in the fact that they cannot be freely converted to and from each other."""

'fundamentally different from [...] subrange types' - what I said above.

'cannot be freely converted to and from *each other*' - what they mean by that
is that

type y = (red, green, blue);
type day = (mon, tue, wed, thur, fri, sat, sun);
var
  color: y;
begin
  color:= fri;
end.

will not work. I don't think anyone would want that ;-)

In any case, we have mode ISO for being extra-ISO-compatible - there are some
significant differences between Borland Pascal and ISO/IEC already. Probably
that mode should also receive the "non-bindable" limitation you cite from
IEC10206. I just noticed: case..else should be a syntax error there,
it doesn't exist in ISO7185 and should be case..otherwise in IEC10206 - where it
is technically mandatory, because a non-matching argument is a dynamic-violation
(RTE).

We also have modes TP and Delphi, and at least there it is *not* an error to
have an unnamed value in a variable, because (spoken in terms of the ISO) the
ordinal-type of an enumerated-type *is* the base type, not a (potentially
non-consecutive) subrange. I've quoted the relevant parts of the language
references multiple times already.
Low/High (and the compiler-internal analogue of them - cf. function getrange()
in FPC) produce the first/last element, but that's it - for example Pred/Succ
may produce unnamed elements.

type
 TT = (a=2,b,c=7,d,e);  // defines constants of type TT for 2,3,7,8,9
{$R+}
var
  t: TT;
begin
  t:= b;
  t:= succ(t);
  Writeln(ord(t)); // writes '4'
end.

Note that FPC doesn't accept this code in mode (Obj)FPC, but correctly does so
in DELPHI, with the same result as Delphi.


Added after Ondrej's message 20:52: Borland appears to have taken the route of
what he called a 'LOW-LEVEL enumeration' from the very beginning.


Martok




___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] Dangerous optimization in CASE..OF

[Martok]

> And you also have subranges of enum types. Can any assumptions made 
> about those in your opinion?

> Does that mean that you would consider the same transformation of a
> case-statement when using a subrange type as correct? And that putting a
> value outside the range of a subrange into such a variable as a
> programmer error? (as opposed to doing the same with a non-subrange enum
> type?)
Depends on the compiler version sadly :/

Subranges for TP5 are documented as "don't rely on anything at runtime, we only
check compiletime", TP7 documents "outside range is an RTE (independent of $R
state)", Delphi is documented like TP5 again.

My intuition was shaped by learning the language with D4 (and D3 books), but
I've always thought that as weird and makes subranges a bit pointless.

I would think that
type
  TEnum = (a,b,c);
  TSubEnum = a..c;

should have the same semantics, but at the same time they can't if subranges are
strict and enums are not. I see now where you're coming from.
(I'll get back to that example at the end.)

And then there's bitpacked records...


> But I finally understand where the disconnect comes from. I have always 
> thought of enums as more or less equivalents of subrange types, simply 
> with an optional name for the values. You, and indeed the Pascal 
> standards, treat them differently.
Getting back to the terms Ondrej introduced yesterday, I think that "normal"
enums may or may not be High-Level enumerations, but enums with explicit
assigment can *only* be Low-Level enumerations. Can we safely distinguish them
in the compiler? Does it even make sense to add that complexity?

This gets weirder. I think Borland already made that distinction, but... not?
<http://docwiki.embarcadero.com/RADStudio/XE5/en/Simple_Types#Enumerated_Types_with_Explicitly_Assigned_Ordinality>

"""An enumerated type is, in effect, a subrange whose lowest and highest values
correspond to the lowest and highest ordinalities of the constants in the
declaration. [...] but the others are accessible through typecasts and through
routines such as Pred, Succ, Inc, and Dec."""

So that's about the "gaps": they're valid, just unnamed.
But for subranges, they write:

"""incrementing or decrementing past the boundary of a subrange simply converts
the value to the base type."""
So we can also leave the min..max range and transparently drop to the parent
type. This raises in $R+, _but is valid otherwise_. (* This is the exact same
text as in the TP5 langref *)

Logical conclusion from that: a variable of a subrange of a
 1) High-Level enum becomes invalid when we leave the declared enum elements
 2) Low-Level enum remains valid by way of dropping to the base type.
Having both variants in the type system is too complex IMO - although it would
be something where the programmer clearly has to state her intentions.


My initial proposed trivial solution was to keep this undefined (maybe document
the difference to BP), and simply change codegen to be undefined-safe normally
and only undefined-unsafe in -O4. I am, however, no longer so sure if that is
really a good solution.

There has to be a reason why everybody else chose Low-Level enums, except that
it is far simpler to implement, right?


> And it would also require us to conditionalise every future optimisation 
> based on type, in particular separating the treatment of enums from that 
> of integers. That's a lot of (future) work and care to deal with what I 
> still consider to be bad programming.
Delphi optimizes always based on the full-range base type:

type
 TB = (a,b,c,d,e); // Sizeof(TB)=1
 TT = a..e;
var
  t: TT;
begin
  t:= TT(2);
  if t <= e then   // does not get removed
  if Ord(t) <= 255 then// 'Condition is always true'




Martok
___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] Dangerous optimization in CASE..OF

[Martok]

Am 16.07.2017 um 16:34 schrieb DaWorm:
> Does the compiler optimize away the else clause in this case?  Seems to me it
> should not.  At least that isn't the behavior I would expect as a user.
At least it should not do that without telling me. The good thing about case
statements is that they tell me of every other programmer error: missing
elements (if used without else), elements that aren't type-correct, double
elements... but not 'extra' else-blocks.


Am 16.07.2017 um 18:26 schrieb DaWorm:
> Academically that may be true, but in the real world that code wouldn't be
> unreachable.  I write code that deals with communication protocols all the
> time.  I can't control what the other side sends.  I have two choices. Write a
> lot of code to validate each and every element is within the proper range, or
> let the handler for each element, that I have to write anyway, deal with the
> unexpected values.
> It can be worked around by casting all parts of the case to integer, but that
> leads to ugly code.
That is exactly my discovery use case, and is also why I keep calling this a
remote code execution: it breaks on sensible network-facing code in a scary way.

Example code snippet from libOpenPGP:
<https://pastebin.com/2wsfCXfP>
One of the more obvious places, this pattern repeats all over the project, from
data parsing all the way down to simple enum-to-string for logging
(SignatureTypeToStr, PKAlgorithmToStr).

It's just a coincidence this is currently partly safe (none of the *ToStr
functions are!), if I was to add a label with value pkPrivate110 (because I
already have pkRSA) and implement the elliptic curve signatures, we would be
jumping to attacker-controlled memory locations. We currently do that in all of
the *ToStr-functions, instead of executing the else-blocks.

That code is completely unambiguous and well-defined if we assume Low-Level
Enumerations (which, coming from BP, I obviously always have).


Martok
___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] Dangerous optimization in CASE..OF

[Martok]

Am 16.07.2017 um 19:58 schrieb Ondrej Pokorny:
> On 16.07.2017 19:24, Martok wrote:
>> The good thing about case statements is that they tell me of every other
>> programmer error: missing elements (if used without else)
> Off-topic: how can I enable this compiler hint?
Erm, I was referring to the "normal" DFA, ie. for function results or variable
initialization.

type
  TEnum = (one, two);

function GetInteger(A: TEnum): Integer;
begin
  case A of
one: Result:= 1;
  end;
end;

... which for some reason only Warns in -O3, and then it's "wrong" sometimes
too, because DFA assumes that enums are Low-Level enums. That was the other
thread on this list recently.

Yeah. Probably a bad argument, sorry.

Martok
___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] Dangerous optimization in CASE..OF

[Martok]

You (Florian) do realize that it's almost impossible to write a C++ program that
is not technically undefined? Their 'standards' are worse than our
'implementation-defined'.

FWIW, GCC agrees with Low-Level Enums, and given that clang regularly catches
hate when their 'optimizations' break stuff like the Linux kernel again...

g++:
  40058b:   8b 45 fcmov-0x4(%rbp),%eax
  40058e:   83 f8 07cmp$0x7,%eax
  400591:   77 76   ja 400609 <_Z1f5tenum+0x89>
  400593:   89 c0   mov%eax,%eax
  400595:   48 8b 04 c5 d8 06 40mov0x4006d8(,%rax,8),%rax
  40059c:   00
  40059d:   ff e0   jmpq   *%rax

g++ -O3:
  4005a4:   83 ff 07cmp$0x7,%edi
  4005a7:   77 14   ja 4005bd <_Z1f5tenum+0x1d>
  4005a9:   89 ff   mov%edi,%edi
  4005ab:   ff 24 fd 18 07 40 00jmpq   *0x400718(,%rdi,8)

Proving my point that we should aim to be better and safer than C, not worse.
___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] Dangerous optimization in CASE..OF

[Martok]

Am 16.07.2017 um 13:17 schrieb Jonas Maebe:
> Does that mean that you would consider the same transformation of a 
> case-statement when using a subrange type as correct? And that putting a 
> value outside the range of a subrange into such a variable as a 
> programmer error? (as opposed to doing the same with a non-subrange enum 
> type?)
Hold on, there already is a test for that particular question in the language
itself!

 -> Can the type be used as an array index?

---
{$mode objfpc}
type
  TExplEnum = (a=1, b=3, c=5, d=7);
  TEnArr = array[TExplEnum] of Byte;
---
=> Error: enums with assignments cannot be used as array index

Makes sense, after all, what should happen with the gaps? And creating the array
for the entire base type with lots of filler data would potentially be too
memory-consuming.

However:
---
{$mode objfpc}
type
  TExplEnum = (a=1, b=3, c=5, d=7);
  TSubEnum = a..d;
  TEnArr = array[TSubEnum] of Byte;

begin
  WriteLn('SizeOf(TEnArr) = ', SizeOf(TEnArr));
  WriteLn('Low(TEnArr) = ', Low(TEnArr), ', ', Ord(Low(TEnArr)));
  WriteLn('High(TEnArr) = ', High(TEnArr), ', ', Ord(High(TEnArr)));
end.
---
SizeOf(TEnArr) = 7
Low(TEnArr) = a, 1
High(TEnArr) = d, 7
---

That difference was unexpected. At least for me.


In {$mode delphi} (and Delphi), we get the second result for both tests. So
there already is some distinction of enum semantics between modes.



Also:
---
  k:= Pred(c);
---
Error: succ or pred on enums with assignments not possible

---
  k:= Pred(TSubEnum(c));
---
Happily compiles.


So, from the compiler's perspective, we cannot rely on the values of enums with
assignments enough to use them as an index (or count them), but we can do so
with subranges - because the subrange in question is effectively 1..7 and
doesn't actually know (or care) about the enum-ness of its host type.


Huh. Fascinating.
___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] Dangerous optimization in CASE..OF

[Martok]

> OK, I see now: there is a difference between C enums and C++ enums. Your 
> example was about C++ enums. My example was about C enums. The C enums 
> are defined to allow any integer value, whereas C++ enums are strongly 
> typed.
In the pages cited, there's no mention of valid ranges, only that for C++ enums,
you need static_cast<>, and that "In the original C and C++ enum types, the
unqualified enumerators are visible throughout the scope in which the enum is
declared. In scoped enums, the enumerator name must be qualified by the enum
type name." That's just our $SCOPEDENUMS switch.
So it really is undefined and clang takes the unsafe option. Sounds familiar.


> C enums: https://msdn.microsoft.com/en-us/library/whbyts4t.aspx 
> https://www.gnu.org/software/gnu-c-manual/gnu-c-manual.pdf
> C++ enums: https://msdn.microsoft.com/en-us/library/2dzy4k6e.aspx
C# enums:
https://docs.microsoft.com/en-us/dotnet/csharp/language-reference/keywords/enum

"A variable of type Days can be assigned any value in the range of the
underlying type; the values are not limited to the named constants."

I mentioned that in passing before, if we take reference from a C-style
language, we should probably use one that shares more ideas (and the lead 
designer).
___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] Dangerous optimization in CASE..OF

[Martok]

I'll start at the end.

>> There has to be a reason why everybody else chose Low-Level enums, except 
>> that
>> it is far simpler to implement, right?
>
> I don't know, but I still don't understand why on Earth you would want
> them in a strongly typed language.
I see them as extremely useful as enums with explicit assignments, and indeed
not so much with "automatic" enums.
Explicit Enums then become a way to have assignment-safe (not typesafe in the
true sense of the word) constants - the compiler can tell me that trying to
assign a SignatureAlgorithm name to a CipherAlgorithm field is wrong, for
example. {$SCOPEDENUMS} can also improve readability - I have a Canon EDSDK
import that only really works because of that constellation.
Or take the gazillion of uint constants for OpenGL, grouping them in a type by
what they do can show when I confuse similar-sounding constants.

I'll reply to your code examples in a second message because I think you just
uncovered another bug.

> Well, unless of course consider having base types that are not a> multiple of 
> 8 bits (I don't see any definition of what can constitute a> base
type on the Delphi page you linked). Then you would also have to> add overflow
checking for non-multiple-of-byte-sized types. And in this> case, you would
still need to support out-of-range values up to whatever> fits in the number of
bits reserved for said base type, but at least it> would make bitpacking
possible. OTOH, in terms of safety or simplicity> of implementation, little or
nothing would be gained.
I don't think they have bitpacking?
Base type for enums is Byte, Word, Cardinal depending on $Z.
"When you use numeric or character constants to define a subrange, the base type
is the smallest integer or character type that contains the specified range."
and I would assume the enum type for subranges over enums.

And yes, subranges are a bit useless outside of their declarative meaning and in
set construction.
From the manuals it looks like they tried changing that in between TP5 and TP7
(probably because of the Pascal standardisation?) but went back to the old
relaxed solution for Delphi.

Proposal:
Everything stays as it is for 'automatic' enums.
'Explicit' enums internally have the full range of their base type
(get_min/max_value, getrange return the base type's values), except for two
functions: the Low() and High() intrinsics continue to return the first/last
declared element.

I believe this is entirely in the previously undefined part of the language. It
makes no change to automatic enums and aligns explicit enums with Delphi. Having
the range functions like that means we don't have to touch any optimizer code at
all - it gets the correct bounds. Same for range checking code.
Subranges continue to be strict (as the "convex hull" of the enumeration's
declared values, but also covering unnamed values in between), so nothing
changes for arrays or bitpacked records.

How about that?

Martok


___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] Dangerous optimization in CASE..OF

[Martok]

>> And then there's bitpacked records...
> 
> Indeed, and range checking. I mean, if you have the above declarations,
> then what should be the behaviour in the following cases:
Delphi generates no range checking code for enum assignments at all, only for
mutation. All your examples work without error, just maybe not in a way you'd 
want.
I think this is a bug, not any assumption - they do check the range in things
like conditional expressions, just not in proper rangechecks.

> 1)
> 
> {$r+}
> var
>   a,b: TEnum;
> begin
>   a:=tenum(5);
>   b:=a;
> end;
Aliasing should not count as an operation, so no rangecheck code inserted at 
all.

> 2)
> 
> {$r+}
> type
>   tr = bitpacked record
> f: TEnum;
>   end;
> var
>   a: tenum;
>   r: tr;
> begin
>   a:=tenum(5);
>   t.f:=a;
> end;
By transition same as 1, but we should get an integer overflow error because
bitsizeof(r.f) < bitsizeof(a).
In my latest proposal it would matter if TEnum has explicit values or not. If it
has, f would be large enough; if not, there should be an overflow check because
two variables of the same type may suddenly not have the same size.

> (which means that in case 2, enums cannot actually be bitpacked)?
I think only automatic enums can be. IMO:

{$Z1}
bitpacked record
  f: (a,b,c,d);
end; => OK, bitsize 2

bitpacked record
  f: (a=6,b,c,d);=> either error (like use as array index in mode FPC)
end; => or OK, bitsize 8 (because of base type)

TEnum = (a=6,b,c,d); => on its own, bitsize 8
bitpacked record
  f: a..d;
end; => OK, bitsize 4 (values up to d=9)

That would be consistent with
  a) how sets of enums are packed
  b) how a subrange over Integer is bitpacked smaller than Integer.

That is, however, again an occasion where the only real use of the subrange is
in its declarative use.

> 3)
> 
> (does this trigger a range check error in Delphi?)
> 
> {$r+}
> var
>   arr: array[tenum] of byte;;
>   a: tenum;
> begin
>   a:=tenum(5);
>   arr[a]:=1;
> end;
Again, no RC code in Delphi, which would be valid only for auto enums.

Bonus:

4)

{$r+}
type
  tenum = (a,b=3,c);
  ta = array[tenum] of byte;
var
  arr: ta;
  v: tenum;
begin
  for v := low(arr) to high(arr) do begin
arr[v]:=1;
  end;
end.

Still no RC code, but now we can be wrong on both sides. Also the loop happily
iterates over the invalid values 1 and 2.

5)

{$r+}
var
  a: TEnum;
  b: TSubEnum;
begin
  a:=tenum(5);
  b:=a;
end;

That should include RC code, and be an error. Indeed that is what FPC currently
generates, Delphi gets it wrong again. Part of the reason why I think this is a
bug on their part.


Martok

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] Dangerous optimization in CASE..OF

[Martok]

As has just been pointed out to me, we all misdiagnosed that example.

TL;DR: you did test undefined behaviour, only a different one. The example
actually proves that clang and gcc agree with the MSDN article in that (even
simple) C++ enums are low-level.


I have verified that Clang/LLVM generates the same code as GCC for the switch
statement itself. It does correctly check for the maximum jmptable index (via
[sub 0x7;ja]) and only then jumps.

LLVM however does generate an UD2(0F0B) trap instruction for several programmer
errors, such as the control flow reaching the end of a non-void function without
return. *That* is why we get a SIGILL when no switch label matches.
In -O1 and -Os, these debug instructions are removed again.

As predicted, there does indeed appear to be a Clang bug: GCC correctly warns
"control reaches end of non-void function", while Clang only emits the UD2
instruction (so it detected it) and does not print the warning.

Completing the function works as expected:

> *snip*
>   return 2;
> case e8:
>   printf("Hello\n");
>   return 3;
>   }
return -1;
> }
> 
> int main()
> *snip*

Another equivalent solution is to have the return in the switch statement's
default label. No UD2 is emitted then.
It follows that the compiler concludes that the default can be matched, even
when all named elements are listed. Therefore, enum variables may contain
unnamed values. Therefore, C++ enums must be Low-Level. QED.


> "Ungültiger Maschinenbefehl (Speicherabzug geschrieben)" = Invalid opcode 
> (memory dump written).
> Why? Because it does not range check before entering the jump table.
I really should have noticed that. A jump into nonexecutable memory would be
SIGSEGV, not SIGILL.

--
Martok

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] Dangerous optimization in CASE..OF

[Martok]

Am 25.07.2017 um 19:31 schrieb Martok:
> As has just been pointed out to me, we all misdiagnosed that example.
Turns out this is not a new question, there is actually a very thorough
treatment of that very issue on SO:
<https://stackoverflow.com/questions/18195312/what-happens-if-you-static-cast-invalid-value-to-enum-class>

(Continue reading after the mention of CWG 1766, the interesting part is what
"representable range" means)

The simple answer however is: a C++ enum with fixed base type (FPC analogue: any
setting other than {$PACKENUM DEFAULT}), any value of the base type is valid.
C++ enums *without* base type may also have a smaller range than int, but always
a power-of-two. That would mean one could mask with that size (we already
generate the masking sometimes) and make the jumptable larger, saving a Jx at
the expense of a larger table.

So the question boils down to: do we want C-Style Enums to behave like in
C-dialects, or just look like they do?

If we do, there's a very simple solution: by setting a fixed $Z option, the
programmer specifically *chose* Low-Level enums. We could just honour that and
be done with it.

--
Martok

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

[fpc-devel] Use of Enumerations in header translations

[Martok]

Hi all,

so, some time ago, Jonas asked this:

> I don't know, but I still don't understand why on Earth you would want
> them [binary-compatible C-style enums] in a strongly typed language.
To which my reply was: header translations!

I've spent some time on Saturday with ppudump and jq to find out what
enumerations with offsets are used for in the FPC tree itself. (Mainly because I
wanted to know what in your opinion the intended and correct application is - I
still *really* want to fix that and get back to it every couple of days.)

The results are interesting: use in packages is exclusively header translations
and on-disk file formats.
Use in the compiler itself is widespread too: the TRegister range type is an
enum where the whole point is that it can contain any value of the base type.
tmsgstate is an enum (with jumps). Binary constants for DWARF are enums.
tinlinenumber is an enum.

All of which I have been told in no uncertain terms are not intended to be safe
and should just not be done.

Many people clearly thought otherwise at some point.

A really good and short header example of the clean and very Pascal-y code
possible by translating enums as enums is the NVAPI header. One instance of our
original issue I could quickly Google up is something like this:
--
var bt: NV_GPU_BUS_TYPE;
NvAPI_GPU_GetBusType(hPhysicalGPU, bt);
case bt of
  NVAPI_GPU_BUS_TYPE_UNDEFINED   : WriteLn('Driver doesn''t know hardware!');
  NVAPI_GPU_BUS_TYPE_PCI : WriteLn('You have a PCI card!');
  NVAPI_GPU_BUS_TYPE_AGP : WriteLn('You have an AGP card!');
  NVAPI_GPU_BUS_TYPE_PCI_EXPRESS : WriteLn('You have a PCIe card!');
  NVAPI_GPU_BUS_TYPE_FPCI: WriteLn('You have an FPCI card!');
else
  WriteLn('You have some future interface!');
end;
--
Looks perfectly sane, right?
NV have in the meantime extended the upstream C enum to include
NVAPI_GPU_BUS_TYPE_AXI = 5. We have an else statement to catch that, but the
compiler may choose to not even generate code for it. This is bad.

Of course, AndiH wrote that header first for Delphi, where it is reliably 
safe...


Why am I posting this? Well, if it is really your intention that enums are only
safe to read if all writes happen in Pascal code without casts, then all of
those imports are plain and simple wrong, and must be changed. If enumerated
types can only formally contain the values explicitly named, then the entire
codegen is undefined.
Or we could just agree that C-style enums must be treated as low-level, which
some committers obviously assumed anyway.


Affected packages visible from win32 target:
==
compiler itself (see type names above)
rtl (windows.pp)

cairo
fcl-db
fcl-sdo
fcl-xml
fftw(very minor)
gtk1(with more affected
gtk2 code in the LCL)
httpd22
httpd24
libcurl
libenet
libusb
libvlc
libxml2
mad
mysql
nvapi
oracle
winunits-base
winunits-jedi
zorba


--
Martok


___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] Dangerous optimization in CASE..OF

[Martok]

Hi all,

another few months, and something funny happened this morning: a tweet shows up
in my timeline which links to this article on efficient codegen for dispatch
with switch statements:
<http://www.cipht.net/2017/10/03/are-jump-tables-always-fastest.html>
Very interesting!

Why I'm resurrecting this thread is the reference to Arthur Sale's 1981 paper
"The Implementation of Case Statements in Pascal". He compares linear lists,
jumptables, binary search and masksearch (on B6700 machines). The bit on
jumptables (journal-page 933) contains this part:
"""The jump-table itself consists of half-word (3 bytes) unconditional branches,
and must be half-word synchronized. The range-check and indexed branch add up to
23 bytes of instructions on the assumption that the range limit values are
fitted into 8-bit literals, and there is a 0-2 byte padding required to achieve
jump-table synchronization. *The range check is never omitted as the
consequences of a wild branch which lands outside the jump-table are potentially
disastrous.* If the range is r, the space requirements are therefore [...]"""

"potentially disastrous" probably didn't mean security as much as "my room-sized
mainframe crashes", but the point stands...

-- 
Martok

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] rdtscp

[Martok]

Hi,

Am 21.10.2017 um 21:41 schrieb Wolf:
> rdtsc cannot do it either. You need to have a CPU capable of 
> understanding rdtscp.
From what I understood, that doesn't give you cycles either, but only the same
timestamp intervals RDTSC returns.
Tangent: On Windows, RDTSC is wrapped by the QueryPerformanceCounter() call. QPC
incidentally is complicated enough that it is very likely no out-of-order
instructions are pending by the time it gets to actually executing RDTSC, but
with less jitter than "abusing" CPUID for serialisation (QPC takes a rather
stable 22 cycles). /Tangent

What I do to get cycle-accurate counts in microbenchmarks is first calibrate the
timer with a known-cycle-length task, obtain a "timestamps per cylce" from that
(depending on core clock, usually between 200 and 1300), and then measure the
actual function. Note that depending on how well you can control multitasking,
interrupts and power management, you will need thousands to millions of repeats
of the function under test to be reasonably free from artefacts.
Looks a bit statistical, but it's precise enough to actually see the instruction
cache, instruction alignment and branch prediction at work. I can also validate
Agner Fog's Instruction Timing Tables with it, so it can't be that bad ;-)

-- 
Regards,
Martok

Ceterum censeo b32079 esse sanandam.

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] x86_64.inc CompareByte

[Martok]

Using the code given below as "inner", I measure this:

Current Trunk:
O0 compare-byte-1 : 196065.112 +/- 896.754 cycles/inner [0.5 %CV 1.6 %R]
O1 compare-byte-1 : 196510.158 +/- 577.976 cycles/inner [0.3 %CV 1.1 %R]
O3 compare-byte-1 : 187540.922 +/- 706.167 cycles/inner [0.4 %CV 1.5 %R]
Patch from 2017-10-21:
O0 compare-byte-2 : 175831.632 +/- 965.972 cycles/inner [0.5 %CV 2.1 %R]
O1 compare-byte-2 : 176039.560 +/- 527.141 cycles/inner [0.3 %CV 1.0 %R]
O3 compare-byte-2 : 158527.167 +/- 661.690 cycles/inner [0.4 %CV 1.5 %R]
(%CV: coefficient of variance * 100%. %R: span as % of mean)

CPU:
 Intel(R) Core(TM) i5-4200M CPU @ 2.50GHz Family 6 Model 60 Stepping 3 (Haswell)
 true single core clock (measured) 2.83 GHz


So the new version is a bit faster, but not by a large margin (10-15%). It is
statistically significant though.
While I'm at it, i386 could use some love:
O1 compare-byte-1 :  755247.183 +/- 8125.671 cycles/inner [1.1 %CV 4.5 %R]
That's 3.8 times slower than x64 for exactly the same code.

Code:
len:=random(100);
for j:=0 to len-1 do
  begin
buf1[j]:=random(256);
buf2[j]:=random(256);
  end;

for j:=0 to random(10) do
  buf2[j]:=buf1[j];

for j:=1 to 1 do
  CompareBytePatch(buf1,buf2,len);  // or System.CompareByte


-- 
Regards,
Martok

Ceterum censeo b32079 esse sanandam.

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] FillWord, FillDWord and FillQWord are very poorly optimised on Win64 (not sure about x86-64 on Linux)

[Martok]

Am 01.11.2017 um 05:58 schrieb J. Gareth Moreton:
> So I've been doing some playing around recently, and noticed that while 
> FillChar has some very fast internal 
> code for initialising a block of memory, making use of non-temporal hints and 
> memory fences, the versions 
> for the larger types fall back to slow Pascal code.
It might be worth it to look at the Pascal versions from generic.inc first, and
see if it is possible to come up with versions that generate faster code.

I'm actually surprised "REP STOSD" should be that much faster. I remember it
being slower on modern platforms than it used to be?

-- 
Regards,
Martok

Ceterum censeo b32079 esse sanandam.

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

[fpc-devel] Question about $OPTIMIZATION LEVELn

[Martok]

Hi all,

just a quick question: is setting one of the LEVELn values like
  {$Optimization LEVEL3}
in a source file supposed to be functionally equivalent to calling the compiler
for that file with -O3?

If so, then there's a bug: only the level gets applied, none of the associated
switches (i.e. level3optimizerswitches, including the inherited lower levels)
are set. This seems weird?
Patch seems straightforward, I'd just want some input on what the negative
options like {$Optimization NOLEVEL3} should do.

Otherwise it's a documentation bug:
<https://www.freepascal.org/docs-html/prog/progsu58.html> claims {$OPTIMIZATION
ON} is equivalent to {$OPTIMIZATION LEVEL2}, which it never was since that
directive was introduced.


Slightly related, the Programmer's Guide lists ancient optimization switches in
11.3. Bug report coming in shortly.

-- 
Regards,
Martok

Ceterum censeo b32079 esse sanandam.

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] Question about $OPTIMIZATION LEVELn

[Martok]

> No - see the discussion at https://bugs.freepascal.org/view.php?id=25873
Huh.

Quick googling shows lots of projects out there that clearly assumed it would.
So we need to get at least the documentation fixed for the future.

If I were to replace {$OPTIMISATIONS ON} with {$OPTIMISATIONS LEVEL4} somewhere,
that switches most optimizations *off* instead of doing more. Not terribly
intuitive.

On the other hand, there is no way to say "do all medium-expensive optimizations
you know on this code": listing them individually is cumbersome, not portable
and not extensible for when new flags are introduced.

Is there even a use case for the current behaviour? I.e., when would one
actually mean -Oolevel3 instead of -O3?


-- 
Regards,
Martok

Ceterum censeo b32079 esse sanandam.

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] Question about $OPTIMIZATION LEVELn

[Martok]

Am 17.01.2018 um 16:38 schrieb Jonas Maebe:
> The LevelX switches are only intended for internal use by the compiler, 
> and were never intended to be exposed.
Good, that's what I thought too.

> The ability to have the equivalent of 
> -O1/-O2/-O3/-O4 in source code was never planned.
Florian wrote in 0025873,
> But there is indeed no equivalent for -O3/4 at source level ({$optimizations 
> on} equals to -O2). I'am open for suggestions how such a directive should 
> look like. 

May I propose to redefine {$optimization levelN} to do that? Reasons:
a) we agree the flags alone (should) have no use
b) it's always been documented that way
c) it's used as if that was true in live code

Rough patch attached...

-- 
Regards,
Martok

Ceterum censeo b32079 esse sanandam.
Index: scandir.pas
===
--- scandir.pas (revision 37943)
+++ scandir.pas (working copy)
@@ -1039,17 +1039,20 @@
 current_scanner.skipspace;
 { Support also the ON and OFF as switch }
 hs:=current_scanner.readid;
-if (hs='ON') then
-  current_settings.optimizerswitches:=level2optimizerswitches
-else if (hs='OFF') then
-  current_settings.optimizerswitches:=[]
-else if (hs='DEFAULT') then
-  current_settings.optimizerswitches:=init_settings.optimizerswitches
+case hs of
+  'OFF': current_settings.optimizerswitches:=[];
+  'ON': current_settings.optimizerswitches:=level2optimizerswitches;
+  'DEFAULT': 
current_settings.optimizerswitches:=init_settings.optimizerswitches;
+  'LEVEL1': 
current_settings.optimizerswitches:=level1optimizerswitches;
+  'LEVEL2': 
current_settings.optimizerswitches:=level2optimizerswitches;
+  'LEVEL3': 
current_settings.optimizerswitches:=level3optimizerswitches;
+  'LEVEL4': 
current_settings.optimizerswitches:=level4optimizerswitches;
 else
   begin
 if not UpdateOptimizerStr(hs,current_settings.optimizerswitches) 
then
   Message1(scan_e_illegal_optimization_specifier,hs);
   end;
+end;
   end;
 
 procedure dir_overflowchecks;
___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] End of support for Win XP?

[Martok]

Am 01.02.2018 um 14:08 schrieb Denis Kozlov:
> It still feels *very early* to drop support for Windows XP. I haven't used it
> properly in years, but I can't say the same about the target user audience. I
> still test some builds against Windows XP.

This may be a very stupid question, but please believe me it is not as troll-y
as it might sound...

What would be the point of supporting older CPUs, if we don't actually support
the OS that runs/ran on them? Or the other way around, there'd be no point in
generating code compatible with a specific processor if the minimum supported
*OS* doesn't run on that cpu.
I'm specifically not talking about the compiler itself (or Lazarus), just being
able to target a platform. That mostly limits the RTL I guess.

I still support Win2000 with one application (with just a few compat shims), and
used to have a Delphi 4 install specifically for one (industry) machine running
NT4 at a previous gig. At my current one, we have a Win95 box running expensive
hardware. True, I ported that program to Win32 last year (Did you know BC++ 5
runs on Win10? I didn't.), but still... that stuff is not as rare as one might 
hope.

What I'm trying to say: it would be amazing if there was a way to be
forward-compatible without entirely scrapping old stuff. Maybe do the same as MS
does, and have IFDEFs for the API level in the headers? I wouldn't mind having
to recompile the RTL.


By the way: Dropping XP support also drops ReactOS support. I'm not sure if we
ever officially had it, but it is on the Wiki.


Martok

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] Redundant compiler hints

[Martok]

The hint was added in response to 
<https://bugs.freepascal.org/view.php?id=31717>.

Not very useful, other than to demonstrate 'inline' is rarely respected.
However, it probably wasn't the intention to generate it for intrinsics as well?

-- 
Regards,
Martok

Ceterum censeo b32079 esse sanandam.

Am 25.03.2018 um 11:56 schrieb Ondrej Pokorny:
> Yesterday I updated FPC to current trunk (r38623) and now I get a lot of 
> redundant compiler hints. E.g.:
> 
> program Test;
> uses SysUtils;
> var
>    FS: TFormatSettings;
> begin
>    FS := DefaultFormatSettings;
> end.
> 
> Test.lpr(6,3) Hint: Call to subroutine "procedure 
> $fpc_copy_proc(Src:Pointer;Dest:Pointer;TypeInfo:Pointer);" marked as 
> inline is not inlined
> 
> Anybody knows what's going on? I am on Windows, use the 32bit compiler.
> 
> Ondrej
> 
> ___
> fpc-devel maillist  -  fpc-devel@lists.freepascal.org
> http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
> 



___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] Wrong docs: not initialized global variables

[Martok]

Am 05.04.2018 um 08:35 schrieb Michael Van Canneyt:
> If the compiler devs wanted, they could initialize every string with the
> '' constant,
That is in fact the -gt option.

> Pascal states: do not assume that variables are initialized.
Corollary: there is no guarantee that "class operator Initialize()" is ever
called, and Maciej can roll back management operators. I very much doubt this is
what you want to imply.
To preempt the argument - no, this is not a different case. Managed types are
initialized at the latest when they come into scope, period.

> From this rule, it follows that every variable must be explicitly initialized 
> [...]
> Be it with an assignment or an 'var a: type = someonstant;'.
... for which the syntactic sugar was rejected not two weeks ago.



To the people referring to Borland Language Guides: As I painfully discovered,
the Borland guides are *not* considered normative material for FPC by certain
FPC developers. Not even for -Mtp.
For some reason, ISO 7185 *is* a reference, even for decidedly not-ISO modes.

Am 05.04.2018 um 12:47 schrieb Alexander Klenin:
> Allow me to yet again to single out this philosophy of
> strongly preferring abstract purity to concrete user experience.
If it at least was consistent purity!

Sorry. Needed to be said.


-- 
Regards,
Martok

Ceterum censeo b32079 esse sanandam.

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] Dangerous optimization in CASE..OF

[Martok]

Am 13.04.2018 um 12:52 schrieb Ondrej Pokorny:
> I introduced the AS operator for enumerators in
> https://bugs.freepascal.org/view.php?id=33603
I'm still not convinced that cementing the FPC-ism of Ada-style high-level enums
is a good idea (and how that is even logically supposed to work with assigned
values), but if we want to go there, something like this feature is absolutely
required (Ada has it).

In that case, off the top of my head, succ/pred, for, bitsizeof and maybe sizeof
need to be fixed as well.

-- 
Regards,
Martok

Ceterum censeo b32079 esse sanandam.

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] Dangerous optimization in CASE..OF

[Martok]

Regarding things to be documented in Delphi: As was established last year, this
aspect of FPCs type system is intended to be a bit more strict than that of the
Hejlsberg languages (*), so I'm not taking anything for granted any more. (Case
in point, while I see it: the very different role of range checks on subranges.
The "Percentile" example a bit down from your link is completely invalid in FPCs
type system, but valid *and* failing rangechecks ever since TP5).

(*) Referring to TP, Delphi, and C# - which all share the same characteristics.

Am 13.04.2018 um 21:21 schrieb Ondrej Pokorny:
> Why? I don't think so. Enums with assigned values are documented to be valid 
> in
> the whole range Low..High:
> 
> http://docwiki.embarcadero.com/RADStudio/Tokyo/en/Simple_Types_(Delphi)#Enumerated_Types_with_Explicitly_Assigned_Ordinality
Most obvious issue: do for loops iterate over the holes in enums?

But I wasn't just referring to enums with assigned values. Even simple enums are
not as simple, as soon as they formally don't fall back on their host type.

type
 enu = (aa, bb, cc, dd);
var
  v: enu;
begin
  v:= dd;   // doesn't matter how dd ends up in v
  v:= succ(v);  // always invalid, no need for {$R+}!
end.

All this is not quite as easy to get right as it seems on the surface. But I do
like the the "v is TEnum" operator from the other side of this thread,
regardless of where this goes...

-- 
Regards,
Martok

Ceterum censeo b32079 esse sanandam.

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] Dangerous optimization in CASE..OF

[Martok]

Am 15.04.2018 um 11:09 schrieb Ondrej Pokorny:
> Please note that I want to support all ordinal values on the left side 
> of the operator. You can then validate a value of variable of the enum 
> type itself.
Useful indeed! It still strikes me as weird that one will need to add this check
many times when porting code from Delphi, but at least now one can.

Jonas will probably argue that this is a tautology and should always evaluate to
true because a variable of a type by definition is of that type, regardless of
content, but it might be considered as somewhat in line with how "var is class"
handles nil values as "not an instance of class".

-- 
Regards,
Martok

Ceterum censeo b32079 esse sanandam.

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

[fpc-devel] Type Compatibility: dynarray <=> Pointer

[Martok]

Hi all,

I have started debugging 0031215, and discovered something slightly unrelated,
but odd. I hope someone can clear that up.
While testing, I found out that the following compiles, and becomes a call to
fpc_dynarray_assign:

var
  arr: array of byte;
begin
  arr:= Pointer(42);
end.

This eventually happens because of the conversion case arraydef<=pointerdef in
defcmp.pas:1151, where a comment says: "nil and voidpointers are compatible with
dyn. arrays". This comment was there since the very first SVN import.

My question: why *are* voidpointers assignment compatible to dynarrays? It does
seem to be Delphi compatible, but I couldn't find any mention in either
documentation that this is possible - only the reserved/constant "nil" is
compatible, and handled elsewhere.

-- 
Regards,
Martok


___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] Type Compatibility: dynarray <=> Pointer

[Martok]

Am 17.04.2018 um 01:51 schrieb Thorsten Engler:
> And the nil assignment variant is pretty much ubiquitous in any code 
> involving dynamic arrays that I'm aware of. 
Yes. I know ;-)

>> only the reserved/constant "nil" is compatible, and handled elsewhere
I asked specifically about assigning arbitrary untyped pointers.


___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] Type Compatibility: dynarray <=> Pointer

[Martok]

Am 17.04.2018 um 11:52 schrieb Stefan Glienke:
> FWIW this has been changed in Delphi 10.2 and does not compile anymore but 
> gives:
> E2010 Incompatible types: 'Dynamic array' and 'Pointer'
That is what I would have expected.
Could you please test what the following does in D10.2?

var arr: array of byte;
begin
  arr:= Pointer(0);
end.

What I'd like to know: do they check the "nil" literal, or pointers of value 
nil?

-- 
Regards,
Martok


___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] Dangerous optimization in CASE..OF

[Martok]

Am 20.04.2018 um 21:35 schrieb Ondrej Pokorny:
> Sven (or anybody else), could you please comment on
> https://bugs.freepascal.org/view.php?id=33603 ? I feel I am getting crazy.
That bug became a textbook example of Cipolla's fourth law, sorry.


> From what I read, there seems to be a difference between FPC and Delphi
> understanding of "enums with holes":
Not even that, the two documentations are not exclusive. Nothing about assigned
enumerated types as documented by FPC excludes Delphi's rules. Which is why I
never understood that:

> Also, the difference is demonstrated in the fact that Delphi and FPC delphi 
> mode allow to define an array of Size. OBJFPC mode doesn't allow it.
It may have something to do with typeinfo (see #27622).
For fpc_read_text_enum et al., a different table is generated, which is correct
even for assigned enumerated types.

> An enumerated type is, in effect, a subrange whose lowest and highest values
> correspond to the lowest and highest ordinalities of the constants in the
> declaration. [...] Only three of these values have names, but the
> others are accessible through typecasts and through routines such as Pred, 
> Succ,
> Inc, and Dec.
The same is true in TP, and by implementation in FPC.

> Therefore I enabled the IS/AS operators on enums with holes only in Delphi 
> mode
> and disabled them in all other modes.
Please do not do this. The other modes are the only ones that really need it, in
Delphi mode, nothing should break on invalid enum values anyway (if it does, it
is a compiler error).


-- 
Regards,
Martok

Ceterum censeo b32079 esse sanandam.

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] Dangerous optimization in CASE..OF

[Martok]

Am 21.04.2018 um 16:40 schrieb Alexander Grotewohl:
> To be honest I agree with what he's saying. We're bending enums to do 
> things normal people just wouldn't (and shouldn't) do.
No, we don't.

There are two semantically different sorts of enums: if you want an enum to
contain only the named elements and nothing else, just use a simple enum without
assigned values. That tells the compiler that you don't really care about
numerical values (except that they are ordinal), the named atoms matter to you.
That implies that the in-memory storage may be opaque.
This is the sort of Enums languages like Ada or Oberon have, and that we inherit
from ISO and TP modes (although they were explicitly documented in TP as
internally being the second type!).

Using assigned values tells the compiler that you *do* care about the ordinal
values. This means that we now need a specific host-type behind it (one that
contains at least all the integers you assigned). Delphi and FPC use a subrange
of the type indicated by $PACKENUM (like a minimum envelope), C-Style languages
use one of the basic integer types as that host-type (because they don't have
subranges). In any case, all values of the host-type are now valid values of the
enum, not just those that have names.
You might think of it like that:
type
  TProgrammerType = (tpJava=2, tpVisualC=4, tpDelphi=100,
 tpClang=101, tpVB=255);
could be roughly rewritten as:
  TProgrammerType = record
  public type
OrdinalType = 2..255;   // sizeof(OrdinalType) := $PACKENUM
  public const
tpJava= OrdinalType(2);
tpVisualC = OrdinalType(4);
tpDelphi  = OrdinalType(100);
tpClang   = OrdinalType(101);
tpVB  = OrdinalType(255);
  end;
You'll notice that if you compile that code normally,
sizeof(TProgrammerType.OrdinalType) would be 1. The storage requirement is
enforced separately from the formal type system, which is essentially the issue
at the very root of this rather long and windy thread.
This would look a bit more obvious in C#, where you might write:
  enum ProgrammerType : ushort = {tpJava=2, tpVisualC=4, tpDelphi=100,
 tpClang=101, tpVB=255};
This syntax makes it very clear that ProgrammerType is a redefinition of ushort,
with some typed constants. The literal translation to plug in to the code above
could be
  public type
OrdinalType = type Word;
Otherwise this is the same.


I hope this clears things up a bit.

-- 
Regards,
Martok

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

[fpc-devel] Debugging Loop Unroll Optimization

[Martok]

Hi all,

as we have discovered in 0033576 and 0033614, there is a bug somewhere in the
loop unroll optimization pass that only appears in complex enough code. The
problem is, it doesn't happen in the single testsuite test, and the observable
crashes in Lazarus are caused by memory corruption at some point unrelated to
the crash, so it's hard to debug (at least on Windows, without rr...).

I have one other project that has sporadic crashes with -OoLOOPUNROLL (I wish I
had figured that out back then), but that is about as difficult as Lazarus,
where it's at least 100% reproducible.

Does anyone have more complete test cases, or maybe smaller affected projects?

-- 
Regards,
Martok

Ceterum censeo b32079 esse sanandam.

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] Debugging Loop Unroll Optimization

[Martok]

I think I found something, see below. Answering the question first ;-)

Am 16.05.2018 um 19:20 schrieb Florian Klämpfl:
> How big is the project? Normally, the number of unrolled loops is reasonable 
> small so comparing the
> generated assembler with and without unrolling should be feasible.According 
> to cloc, 40kLOC, 10kLOC of which are DSP code, not counting packages
that are not in the source repo. With the very spurious crashes, not the best
test case...

I did, however, do the insane thing and diff Lazarus (or rather, IDAs output).
There are a lot of unrolled loops in the LCL alone, but the code reordering for
the first 15% or so that I checked looks fine. {Off-Topic: could parts of that
variable rewriting be used for more aggressive inlining?}


Some more testing with Lazarus and -gt has shown that at some variables are
uninitialized with -Ooloopunroll. There is a fairly common case where removing
the variable is not good:
-
for Result:= alow to ahigh do
  if Something(Result) then exit;
-
If that gets unrolled, Result is undefined, and whatever it gets assigned to
receives the random content of r/eax.

I'm 90% sure that global variables and the Result variable are *not* supposed to
be undefined after the loop (in contrast to Delphi-compatible locals), but can't
seem to find a reference for that. This pattern is used all over the place, so
somebody else must have thought so too? Grepping for "for result:=" over the FPC
source tree gives 10 matches, Lazarus has over 100 results.

Test case:
-
{$Optimization LOOPUNROLL}

var
  i : integer;
  s : single;

function tfun: integer;
begin
  for Result:=1 to 10 do
s:=s+1;
end;

begin
  s:=0.0;
  for i:=1 to 2 do
s:=s+1;
  if i <> 2 then
halt(1);
  i:= tfun();
  if i <> 10 then
halt(2);
  if s <> 12 then
halt(3);
  writeln('ok');
end.
-

I would very much expect that to be the main cause of the observed crashes.

--
Regards,
Martok


___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] Debugging Loop Unroll Optimization

[Martok]

Same documentation for FPC, 
<https://www.freepascal.org/docs-html/ref/refsu58.html>

But, someone clearly explicitly thought otherwise at some point:
<https://svn.freepascal.org/cgi-bin/viewvc.cgi/trunk/compiler/aasmcnst.pas?view=markup#l860>
That warning is only "wrong" if Result is well-defined after exiting the loop.

> So this shall not be unrolled, but is still error prone, if Result is not set 
> after the regular end
> of the for loop.
Am I doing something wrong, or does FPC not emit a warning when a loop variable
is read after the loop? Delphi has a warning, and does indeed display it for the
test case. There is no indication anything might be wrong from FPC?

This kind of code is used even more than "that other thing", makes me wonder if
it's a good idea to break this at O3...

-- 
Regards,
Martok


___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] Debugging Loop Unroll Optimization

[Martok]

> Can you report that to the bugtracker of Lazarus?
Sure. Done as https://bugs.freepascal.org/view.php?id=33753

Martok

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] Debugging Loop Unroll Optimization

[Martok]

Am 17.05.2018 um 21:06 schrieb Florian Klämpfl:
> -O3 probably "breaks" a lot of code which exploits the limits of a language-
True.

I do wonder if it would be good to create some sort of collection of things that
are undefined in the core language and how different compilers and dialects
handle them. It's been 28 years, there's probably consensus on some out there.
Could be useful as a guide for porting code as well.


Martok

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] Debugging Loop Unroll Optimization

[Martok]

Am 18.05.2018 um 14:07 schrieb J. Gareth Moreton:
> What's the easiest solution to this? Change the code or change the rule?
Follow consensus. There's little point in ignoring 30 years of language
development, just because nobody started an ISO committee.

In this case: """After a for-statement is executed, other than being left by a
goto-statement, the control-variable shall be undefined"""
Break/continue/exit wasn't even invented then!

Borland extended this, hence the VCL samples mentioned.


But that is a discussion I have wasted a lot of time on once already, no
intention to do that again.

-- 
Regards,
Martok


___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] Debugging Loop Unroll Optimization

[Martok]

> Citation: "If the loop was terminated prematurely with an exception or a 
> break statement, the loop variable retains the value it had when the 
> loop was exited."
As a quick fix, not unrolling loops left with exit at least fixes this specific
situation. This still leaves exceptions raised, but IIRC the handlers don't
restore context anyways, we might be okay?

diff --git a/compiler/optloop.pas b/compiler/optloop.pas
index 46039ffc5a..dc714ea2cc 100644
--- a/compiler/optloop.pas
+++ b/compiler/optloop.pas
@@ -76,7 +76,7 @@ unit optloop;

 function checkbreakcontinue(var n:tnode; arg: pointer): foreachnoderesult;
   begin
-if n.nodetype in [breakn,continuen] then
+if n.nodetype in [breakn,continuen,exitn] then
   result:=fen_norecurse_true
 else
   result:=fen_false;

I'll be running this on today's snapshot, see if anything else remains.

-- 
Regards,
Martok


___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] Debugging Loop Unroll Optimization

[Martok]

> Sorry to waste your time on this.
Don't worry, I like investigating this stuff. I don't like the rule-lawyering
that usually follows ;-)

>  I'm glad it states the for-loop variable will be left undefined - that's 
> good enough documentation for me.
It is *not* undefined when the loop is left with break or exit, that's the
point. The ISO is not a very good reference for modern-ish Pascal.

-- 
Regards,
Martok

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] Debugging Loop Unroll Optimization

[Martok]

Am 18.05.2018 um 17:15 schrieb Sven Barth via fpc-devel:
> Maybe it should also check for goto and at least explicit raise statements? 
That'd be adding goton and raisen to the check, right?

For now, just checking exit seems to be enough to get rid all of the easily
testable Lazarus crashes. The "Optimized IDE" profile is usable again. I'll post
the patch to the bug tracker, as we seem to have decided it is not a hack ;-)

-- 
Regards,
Martok


___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] Debugging Loop Unroll Optimization

[Martok]

Am 21.05.2018 um 17:44 schrieb Florian Klämpfl:
> I added raise, exit, goto and label as well.
Oh, label, right.

I'd say #0033614 can be resolved as "fixed in 39083" and #0033753 as "no change
required" then.

-- 
Regards,
Martok


___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] Debugging Loop Unroll Optimization

[Martok]

Am 21.05.2018 um 19:27 schrieb Ondrej Pokorny:
> Well there is still something left:
You are right, variables don't correctly get captured (or rather, don't get
captured at all). This is not good.
Dirty fix: only unroll loops over local variables when there are no nested
procs. But that probably prohibits most useful cases as well...

I'd say that closures + AST-level inlining + some dead store eliminations would
fix a lot of issues that currently have special case handling.

-- 
Regards,
Martok


___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] Possible internal corruption

[Martok]

> A clue that leads me to believe there's internal corruption is that a produced
> .s file yields an alignment field of ".balign 119,0x90", which should never
> happen.
Could you set a breakpoint on aggas.pas:721 (the call to doalign) with a
conditional on "tai_align_abstract(hp).aligntype=119" and check what the actual
type of hp is? It could be that at some point a node gets its typ changed in an
invalid way?
aligntype should be either one of 2^[0..5], never something else...

This is where AddressSanitizer support would be *nice*.


-- 
Regards,
Martok

Ceterum censeo b32079 esse sanandam.

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] Managed Types, Undefined Bhaviour

[Martok]

> I hope this issue gets addressed, as I deem the current behaviour completely
> broken and also going totally against the spirit of Pascal, feeling much more
> like some very obscure behaviour I'd expect from some C compiler.
> Discovering the handling of this issue, however, makes me wonder
> whether fpc aims to be a great Pascal compiler that does without bad surprises
> and very very hard to debug "documented" behaviour or not.
There is less undefined behaviour than in C, but the one we have will bite you
in the most awful ways, sometimes after years of working just fine. And we don't
even have a nice formal standards document that one could grep for "undefined".

But yeah, as Jonas wrote, this _isn't_ one of these occasions. FPC uses (and
reuses!) tempvars a lot more than Delphi, which causes all sorts of funny
behaviours with managed types. Try returning a string or use the
JavaScript-style "Foo().Bar().Baz()" method chaining pattern and you'll see what
I mean.

Change your function to the following, and it will do what one would expect:
function DoSomething (len: longint): Vector;
begin
  Result:= nil;
  SetLength (result,len); // whatever
end;

For managed types, as far as I can tell:
 - locals are initialized (even if there is a warning telling you they are not)
 - tempvars are initialized *once*
 - Result is never initialized (there is no warning telling you it is not).


-- 
Regards,
Martok

Ceterum censeo b32079 esse sanandam.

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] Managed Types, Undefined Bhaviour

[Martok]

Am 29.06.2018 um 12:43 schrieb Michael Van Canneyt:
> Out of curiosity, can you give a simple example of such a funny behaviour
> in such a chaining pattern ?
We've had this topic about 2 years ago with regard to automatic file close on
interface release. Interestingly, something must have changed in the mean time,
because the trivial testcase is now *different* , which is somewhat the point of
being weird-undefined ;-)

Take this example: https://pastebin.com/gsdVXWAi

The tempvar used to get reused, causing lifetime issues with the "chain object".
This isn't the case anymore, now three independent tempvars are used, all of
which live until the end of the function, potentially keeping the object alive
for a long time.
There is also one fpc_intf_assign with associated addref/release per as
operator, which isn't technically necessary.

One could probably avoid the interfaces here with ARC records, but either I'm
missing something or the scope lifetime of tempvars there is even worse.

-- 
Regards,
Martok


___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] Managed Types, Undefined Bhaviour

[Martok]

Am 29.06.2018 um 14:41 schrieb Michael Van Canneyt:
> As far as I can see, you get 2 chain and 1 done call. Which is what I'd 
> expect.
> The overrides of the _* calls are useless, since they are not virtual in
> TInterfacedObject and hence never called. So that's OK too.
Interface functions are always virtual and implemented by the actually
instantiated class. The "override" keyword is neither allowed nor needed, this
code does what it looks like.

The expected output would be 3 Addrefs and 3 Releases.
A bit better would be doing the last release after "Done" and before "fin" (but
this is difficult because of the implied exception frame - there are cases where
Delphi does it anyway, depending on method length).
The "ideal" output would get away with even less (but I don't think this is
possible without translating to SSA form first and doing some strict counting).

The observed output is 6 Addrefs and 6 Releases. At some point in the past (this
may have to do with variable and register allocation), a Release could happen
too soon. It doesn't now, so that's good.

There is nothing technically wrong with the generated code, but it is not nearly
as efficient as it could be. See also Ryan's comments about slow
Interlocked*-calls a few weeks ago. Delphi's output for the same example is
better, giving the expected output.
Because of the tempvars, it is also not exactly what one might expect at first,
which is why I mentioned it in this context.

-- 
Regards,
Martok


___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] Managed Types, Undefined Bhaviour

[Martok]

Am 29.06.2018 um 16:05 schrieb Michael Van Canneyt:
>> The expected output would be 3 Addrefs and 3 Releases.
> 
> I don't get that.

Somewhat current FPC trunk output, annotations added manually:
==
Addref: 0022FAA8 Refcount: 1 at 00404961
   (by fpc_class_as_intf in GetChainer)
Addref: 0022FAA8 Refcount: 2 at 00404223
   (by fpc_intf_assign of GetChainer Result)
Release: 0022FAA8 Refcount: 2 at 004041F4
   (by fpc_intf_decr_ref of GetChainer Result)
Chain: 0022FAA8
Addref: 0022FAA8 Refcount: 2 at 00404961
   (by fpc_class_as_intf in Chain)
Addref: 0022FAA8 Refcount: 3 at 00404223
   (by fpc_intf_assign of Chain Result)
Release: 0022FAA8 Refcount: 3 at 004041F4
   (by fpc_intf_decr_ref of Chain Result)
Chain: 0022FAA8
Addref: 0022FAA8 Refcount: 3 at 00404961
Addref: 0022FAA8 Refcount: 4 at 00404223
Release: 0022FAA8 Refcount: 4 at 004041F4
Done: 0022FAA8
fin
Release: 0022FAA8 Refcount: 3 at 004041F4
   (by fpc_intf_decr_ref at scope end of Test)
Release: 0022FAA8 Refcount: 2 at 004041F4
   (dito)
Release: 0022FAA8 Refcount: 1 at 004041F4
   (dito)
==


Delphi output (without the stack trace part, because they don't have it):
==
Addref: 0205DBE8 Refcount: 1
Chain: 0205DBE8
Addref: 0205DBE8 Refcount: 2
Chain: 0205DBE8
Addref: 0205DBE8 Refcount: 3
Done: 0205DBE8
fin
Release: 0205DBE8 Refcount: 3
Release: 0205DBE8 Refcount: 2
Release: 0205DBE8 Refcount: 1
==


Delphi uses a shortcut for "as", and because of their different handling of
putting Result in a tempvar, they get away with less internal assignments.

6 LOCK instructions (and associated calls) less. Not a lot by itself, but since
we're counting single-digit cycles in other places...


-- 
Regards,
Martok


___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] Managed Types, Undefined Bhaviour

[Martok]

Am 29.06.2018 um 16:37 schrieb Thorsten Engler:
> The specific functions that implement an interface get baked into the class 
> at the moment when the interface is defined as part of the class. This 
> results in important differences in behaviour, depending if methods (in the 
> class) are defined as virtual or not, and if a derived class redeclares an 
> interface already declared on an ancestor or not.
Okay, then why does the calling convention change matters so much?

Maybe a COM/CORBA thing? COM interface methods can't logically not be virtual,
and the kind of code from my example has always worked (for me!) in FPC.

I am confused. Which sorta ties in to the whole "surprises" thing from before we
hijacked this thread...

-- 
Regards,
Martok

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] Managed Types, Undefined Bhaviour

[Martok]

Am 29.06.2018 um 21:27 schrieb Thorsten Engler:
>>> [...] COM interface methods can't logically not be virtual,
>>
>> I think you are confusing things here. They can be virtual or not
>> virtual in COM and CORBA.
> 
> I think a lot of people simply don't understand how interfaces are 
> implemented.
Thank you for the explanation! Saved for future reference.

I was thinking too much in terms of C++ pure virtual classes and their VMT and
forgot about the self translation trampoline functions.


-- 
Regards,
Martok


___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] Issue #32913

[Martok]

Am 14.07.2018 um 20:41 schrieb Dmitry Boyarintsev:
> Exiting from finally? Is it something new?0
> 
> http://docs.embarcadero.com/products/rad_studio/delphiAndcpp2009/HelpUpdate2/EN/html/devcommon/cm_cant_leave_finally_xml.html
Still documented for Tokyo:
<http://docwiki.embarcadero.com/RADStudio/Tokyo/en/E2126_Cannot_BREAK,_CONTINUE_or_EXIT_out_of_a_FINALLY_clause_%28Delphi%29>

Although the example is of course wrong, the way it's given would be 'E2097
BREAK or CONTINUE outside of loop'. Just substitute "break" with "exit".

Control flow statements are allowed in an EXCEPT block. Come to think of it,
didn't we have another TRY/FINALLY/EXCEPT nesting issue that might have
something to do with that?

-- 
Regards,
Martok

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] Progress of pure function research

[Martok]

Am 16.07.2018 um 07:01 schrieb J. Gareth Moreton:
> As stated in the Wiki page, my first test case is the Max function.  Since it
> works both as an inline and a pure function, I can easily change the 
> directive to
> analyse the code flow in the compiler.
I may have missed this in the discussion before. But isn't that a prime example
for "simple" const propagation?

==
function Max(a, b: integer): integer; inline;
begin
  if a > b then
Result:= a
  else
Result:= b;
end;

  z:= Max(1, 2);
==
That already gets optimized to `z:= 2;` on -O1, while the following needs -O2,
but gets to the same result:
==
  x:= 1;
  y:= 2;
  z:= Max(x, y);
==

Tail recursion expansion could do the same for certain recursive functions.


-- 
Regards,
Martok


___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] Progress of pure function research

[Martok]

Am 16.07.2018 um 18:54 schrieb J. Gareth Moreton:
> In that situation, yes it is.  Max is a very simple function though, and it 
> gets
> more complicated if it appears in another pure function where the parameters 
> are
> variables, but whose values are deterministic.
Actually, this might be more workable than I thought at first. The "Thing from
-O3" that carries out the simplification in my example (it also works for mixed
things like "x:= 1; z:= Max(x, 2);") is exactly the constant propagation
optimization. It might be enough to simply force a few localized optimizations
after inlining and before simplifying.
I'll have a look into that later - this would be useful for many cases
regardless of pure functions.

-- 
Regards,
Martok


___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

[fpc-devel] Compile time functions

[Martok]

Hi all,

inspired by Kit's concept for pure functions, I had a deeper look at what inline
functions (and more importantly, the nodes involved in expanding them) can do.
Let me share my proof-of-concept (which may very well become a
'poof-of-concept'...). You can find my current working branch here
<https://github.com/martok/freepascal/compare/master...dev_inlining>.

Instead of marking a function as 'pure' at declaration, I took another way: make
normal inlining firstpass flexible enough that any "accidentally" pure function
can be collapsed at the call site.
As an extension, I then propose a new intrinsic:

function ConstExpr(X: AnyType): AnyType;
If the expression X can be reduced to a constant, the node simplifies to
that constant.
Otherwise, fail compilation with E3203 Illegal Expression.

ConstExpr is basically pass-through for most expressions, but it allows us to
"call" functions /at parse time/, as long as these functions are pure and the
arguments are constant at the point of declaration.

Let me illustrate with an example:

program tinline3;

  function sinc(x: Real): Real; inline;
  begin
    if X = 0 then
  Result:= 1
    else
  Result:= sin(x) / x;
  end;

const
  s1 = constexpr(sinc(0.3));
  s2 = constexpr(sinc(0));
var
  u, v: Real;
begin
  u:= s1;
  u:= s2;
end.

This gets processed into the node tree (right at parse time):


***
after parsing
$main; Register;

***
(blockn, resultdef = $void = "untyped", pos = (16,1), loc = LOC_INVALID,
expectloc = LOC_INVALID, flags = [], cmplx = 4
   (statementn, resultdef = , pos = (17,9), loc = LOC_INVALID,
expectloc = LOC_INVALID, flags = [], cmplx = 4
  (assignn, resultdef = $void = "untyped", pos = (17,3), loc =
LOC_INVALID, expectloc = LOC_INVALID, flags = [], cmplx = 2
 (loadn, resultdef = Real = "Double", pos = (17,3), loc =
LOC_INVALID, expectloc = LOC_INVALID, flags = [nf_write], cmplx = 1
    nil
    symbol = U
 )
 (realconstn, resultdef = Real = "Double", pos = (17,7), loc =
LOC_INVALID, expectloc = LOC_INVALID, flags = [], cmplx = 2
    value =  9.85067355537798561294E-0001
 )
  )

   )
   (statementn, resultdef = , pos = (18,9), loc = LOC_INVALID,
expectloc = LOC_INVALID, flags = [], cmplx = 2
  (assignn, resultdef = $void = "untyped", pos = (18,3), loc =
LOC_INVALID, expectloc = LOC_INVALID, flags = [], cmplx = 2
 (loadn, resultdef = Real = "Double", pos = (18,3), loc =
LOC_INVALID, expectloc = LOC_INVALID, flags = [nf_write], cmplx = 1
    nil
    symbol = U
 )
 (realconstn, resultdef = Real = "Double", pos = (18,7), loc =
LOC_INVALID, expectloc = LOC_INVALID, flags = [], cmplx = 2
    value =  1.E+
 )
  )

   )
)

As you can see, the expressions have been simplified into (the correct)
constants of type Double (Real for x86 is Double). Intermediate type conversions
(including potential loss of FP-precision) are handled correctly (there's a
sneaky int(0)<>real in the example).
This is already really useful for the most common application of macros with
parameters in C: calculating constants (like #define _IOC(inout,group,num,len)
(inout | ((len & IOCPARM_MASK) << 16) | ((group) << 8) | (num))).

There are currently a few issues:

 1. I extended the const propagation optimisation pass a bit. It had
restrictions on inlinen and realconstn that probably had a reason, but I
can't see it...
 2. calln now does an automatic internal optconstpropagate pass when inlining.
This is to carry refs to constant temps a bit further down. That might in
fact be problematic because it could increase code size with multiple loads
and should probably not be done in general. But for this PoC, that was the
simplest way.
 3. procs can only be inlined if their body has been fully parsed before the
call. This is actually kind of a problem because it limits usefulness for
the C-style constant-macro application.
 4. The tricky part (same goes for pure functions) is actually finding out if
parameters of an inlined function are constants. optconstpropagate can track
that after the fact, but at that point the fun is already over. The best
solution I could think of is running some very primitive DFA along while
parsing, so that the firstpass can refer to that. Basically, if a symbol is
known to be constant, loads of it could be trea

Re: [fpc-devel] Compile time functions

[Martok]

Am 21.07.2018 um 02:08 schrieb Sven Barth via fpc-devel:
> The main problem is that not all functions that would be eligible for your
> approach are also declared as inline thus their node trees would not be stored
> inside the PPU and thus you could not work with them.
I'm not sure I understand. Isn't the same true, with a new feature? The full
node tree must be available, which basically means "has inlineinfo". Doesn't it?

> Additional benefit of the "pure" modifier: the compiler can check once the
> method has been parsed whether it's pure or not and can thus error out in the
> latter case.
That is true, it is a more explicit declaration of intent.
Maybe the codegen of pure can simply be implemented as generating inline info,
but always replacing the calls with the simplified version. If it is already
known that a function is pure, what I did with constexpr() would basically be
guaranteed to work.

-- 
Regards,
Martok

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] Compile time functions

[Martok]

Am 22.07.2018 um 06:03 schrieb J. Gareth Moreton:
> Pure functions cannot be
> evaluated at compile time if one of the arguments is a variable (or at the 
> very
> least, not deterministic)
Do you have an idea how to prove that yet? The way I see it now, this is the
only hard issue, every other component is already present somewhere in some 
form.

> although there
> might still be issues in exceptional conditions (e.g. division by zero or 
> trying
> to calculate tan(pi/2), for example).
Good point - what should happen in that case?
const
  x  = Pi / 0;
begin
  writeln(x);
end.
That writes "+Inf".

> The other point with a "pure" modifier is that, assuming the compiler 
> determines
> that the function is indeed pure, it would permit their assignment to a 
> constant.
So: on processing a call to a pure function,
  if arguments are constant, emit (interpreted) result of function
  otherwise, emit regular call
?
What would error/warning cases be? I'd want to avoid a situation like that with
the insightful, but unfixable and thus spammy "06058_N_Call to subroutine "$1"
marked as inline is not inlined" message.

> The final issue is that while a function might be pure, you might not want to
> inline it because of its complexity, especially if it's called multiple times
> with variable arguments.
That is very true. Should the "interpretation complexity" be limited, or should
the compiler keep going and maybe run into a stack overflow?


-- 
Regards,
Martok


___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] Why/how does the compiler have a non-trivial number ofmemory leaks after over two decades of development?

[Martok]

Am 30.07.2018 um 14:24 schrieb Marcos Douglas B. Santos:
> Is performance more important than being correct?  :|
In this project, the answer is always taken to be yes.

-- 
Regards,
Martok


___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] Case code pattern

[Martok]

Hi,

> I would need a clarification about the way the case statement is 
> translated into assembler by FPC. When the list of alternatives is 
> continous, does the compiler generate a jump table?
What Kit said, but a correction: the threshold is not 50, it is 19. And what is 
generated is not technically a jump table, but a
typed dispatch table.


-- 
Regards,
Martok

Ceterum censeo b32079 esse sanandam.

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] Case code pattern

[Martok]

Am 14.08.2018 um 11:27 schrieb Marco Borsari via fpc-devel:
>  From what I can read from Wikipedia, every compound of the case is 
> enclosed in
> a procedure (or in a function, as you said the table is typed), declared 
> with
> nostackframe, and called by an array of index, right?

The blocks are just regular labels. What I meant was the jump array index is 
considered typed, which allows some optimizations.
This is noteworthy because FPC is the only compiler I could find that does that.


Roughly:

case x of
  0: code;
  1: Morecode;
  2: EvenMoreCode;
  //... assume we have enough case labels to get a jumptable
end;


Gets translated as if one had written:

label label0,label1,label2,{...,}afterend;
const table: array [lowestcaselabel..highestcaselabel] of CodePointer = 
(@label0, @label1, @label2{,...});
if (xhighestcaselabel) then
  goto @afterend;
goto table[x];
label0:
  code;
  goto afterend;
label1:
  Morecode;
  goto afterend;
label2:
  EvenMoreCode;
{...}
afterend:


Iff it follows from the strong typing of the array index that the "if" at the 
start is always false for all defined values of
the type of x, it is omitted.

-- 
Regards,
Martok

Ceterum censeo b32079 esse sanandam.

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] x86_64 Optimizer Overhaul

[Martok]

Am 12.12.2018 um 04:51 schrieb Ryan Joseph:
> I’ve spent some time in the compiler sources now and I’m curious just where 
> people think the bottle necks for performance actually are. It’s such a 
> complicated system for anyone one person to have a good understanding of so 
> it’s not clear where to begin looking.

Checking out the memory manager(s) could be useful as well - there are a lot of
small allocations, that generally tends to put much stress on it.
And any improvement there would also directly benefit user applications.

-- 
Regards,
Martok


___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] x86_64 Optimizer Overhaul

[Martok]

Am 15.12.2018 um 17:12 schrieb Florian Klämpfl:
> The memory manager itself pools already, so no need for the compiler. If 
> somebody wants to improve the heap manager:
> implement OS supported re-allocations (OS can move memory by just shuffling 
> pages).

Very much agree, it's not a user program's job to work around the standard
memory manager in daily use. Doing that is a C++-ism that shouldn't exist in a
sane environment ;-)

I just tested something, and I'm a surprised by how big the difference is. This
simple test is 1.5 times slower in FPC/trunk/win32 than Delphi 2007 and 2.8
times slower for instances of TComponent. Medium-size GetMemory (I tested 123
bytes) is 22 times slower in FPC.
Looks like there is quite some potential there.


const COUNT=1;
var
  t1, t2: dword;
  objs: array[0..1] of TObject;
  i, j: integer;
begin
  t1:= Gettickcount;
  for i := 0 to COUNT - 1 do begin
for j := 0 to high(objs) do
  objs[j]:= TObject.Create;
for j := 0 to high(objs) do
  objs[j].Free;
  end;
  t2:= Gettickcount;

  writeln((t2-t1)/COUNT:10:3, 'ms');
  Readln;
end.

-- 
Regards,
Martok


___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] x86_64 Optimizer Overhaul

[Martok]

Sorry for hijacking the thread. Your mail client issue makes the conversation
really hard to follow, so I have literally no idea what the current subtopic of
a reply chain is, and there's little point in properly detaching a thread.


Am 15.12.2018 um 18:13 schrieb J. Gareth Moreton:
> I dare ask, does that mean we should avoid workarounds in the compiler (and
> our own programs) that aim to avoid constant construction and destruction of 
> objects, and instead try to improve the memory manager?

I was thinking more along the lines of avoiding cycle-counting special paths at
the cost of reliability, when there are much larger issues that would benefit
every program.

I would not be surprised if some of the large difference Simon listed when
calling out the bounty come from this side, instead of raw instruction 
throughput.


> Thus, I would imagine that Delphi's *default* internal memory management
> system is more along the lines of what is done in FPC's cmem unit, which is
> well known to be objectively faster than FPC's default memory manager
I'm fairly certain the runtime is written in Pascal, except for parts of the
startup code. The memory manager at some point (I think D2006?) adopted FastMM:
<http://docwiki.embarcadero.com/RADStudio/2010/en/Configuring_the_Memory_Manager>
In any case, FPC's cmem on Win32 calls into mscvrt, and that is so slow that I
killed the test code after a couple of minutes, where even FPC-builtin was done
after 10 seconds.



-- 
Regards,
Martok


___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] Case block optimisation

[Martok]

Am 15.12.2018 um 18:08 schrieb J. Gareth Moreton:
> So here's something else I've been playing with in the interim... I've been
> working on improving the algorithms used when compiling case blocks.  It was
> driven partly by my binary search idea for certain kinds of case block, which 
> I
> wrote up over here: http://wiki.freepascal.org/Case_Compiler_Optimization

Feel free to include '0033093_Casenode_order.patch' from
<https://bugs.freepascal.org/view.php?id=33093>
It's actually already a bit simpler in your version ;-)

The one thing that still bugs me (but I know it's not really seen as a problem
in FPC) is that every platform cg theoretically has to implement all of that,
causing a lot of duplication.

I like the use of more expressive intermediate names. But there was a lot of
overflowed-aint-shuffling, are you sure that still works everywhere?

Bug 0032115 provides a "nice" test case for things that can go wrong with
different word sizes, and is also a good test for the true label count.


-- 
Regards,
Martok

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

[fpc-devel] TDef mode question

[Martok]

Hi all,

just a very quick question - I think the answer is no, but that doesn't mean
anything:

Is there a (ppu-persisted) way to find out in what language mode (fpc, iso,
tp..) a tdef was originally declared? The current_settings.modeswitches
obviously refer to the module it is currently used in.


Cheers,
Martok

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] TDef mode question

[Martok]

Am 27.12.2018 um 20:48 schrieb Jonas Maebe:
> No, there is not. If a def needs to be different depending on the 
> language mode, you have to store it in the def.
Thought so. Thank you for confirming!


Cheers,
Martok

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

[fpc-devel] [Patch/RFC] Warnings for (in/over)complete case statements

[Martok]

Hi all,

The attached patch adds two messages inspired by C compiler's -Wswitch-enum and
-Wcovered-switch-default. Building on the recent label count refactoring, this
was fairly straightforward.

- If a case statement on an ordinal does not contain labels for all values of
the ordinal, and no else statement is given, raise a new warning (W6059). This
is actually defined as an error in ISO7185 and a dynamic-violation in IEC10206.

- If a case statement has labels for the entire range of the type, and an (now
never used) else statement is present, a warning "Unreachable code" is raised.

Both cases are clearly something where the compiler and programmer don't agree
on something, hence drawing attention to these situations.

The checks are enabled only for enumerated types and small (1-byte) integers. In
C, they are only for enumerated types, I added small types because they are
often used as tag types, where this check is extra useful.

Now, the RFC part. I have a few open questions...

* does it make sense to do that to all integral types? In ISO mode, it's
formally required for all case statements. I left it out for now as sometimes
case..of is used as a shorthand for multiple in..[range], there would be more
detections that look like false positives.

* in ISO mode, there are 2 solutions: in Standard Pascal, not covering the
entire type is a compile-time error. In Extended Pascal, this is a runtime
error. Which one is followed usually?

* Building the compiler itself cycles with -Sew, so it bails on the first
occurrence of one of these issues - turns out they're all over the place.
However... I think the warnings are correct and expose imperfect code here.
Take this example (one of the first encountered):
<https://github.com/graemeg/freepascal/blob/master/compiler/defutil.pas#L1630>
stringtype is an enum and all values are used. There cannot be any other values
(as we have established last year). Therefore, the else clause is just as wrong
as an "if false then", which would be caught already.

* Adding to the previous, since it is now possible to discover forgotten items
or later additions by the other warning, removing these 'safety' else-clauses
would improve code quality.


What do you think?

-- 
Regards,
Martok

From e32addb6583c8b752c168fe221385566499625bb Mon Sep 17 00:00:00 2001
From: Martok 
Date: Tue, 1 Jan 2019 18:59:56 +0100
Subject: [PATCH] Report warnings for incomplete and over-complete case
 statements

+ regenerated messages
---
 compiler/msg/errore.msg |   5 +-
 compiler/msgidx.inc |   5 +-
 compiler/msgtxt.inc | 816 
 compiler/ncgset.pas |  28 +-
 4 files changed, 440 insertions(+), 414 deletions(-)

diff --git a/compiler/msg/errore.msg b/compiler/msg/errore.msg
index b6448c25ba..449a9a1a4c 100644
--- a/compiler/msg/errore.msg
+++ b/compiler/msg/errore.msg
@@ -2350,7 +2350,7 @@ sym_e_type_must_be_rec_or_object=05098_E_Record or object 
type expected
 #
 # Codegenerator
 #
-# 06049 is the last used one
+# 06059 is the last used one
 #
 % \section{Code generator messages}
 % This section lists all messages that can be displayed if the code
@@ -2505,6 +2505,9 @@ cg_n_no_inline=06058_N_Call to subroutine "$1" marked as 
inline is not inlined
 % The directive inline is only a hint to the compiler. Sometimes the compiler 
ignores this hint, a subroutine
 % marked as inline is not inlined. In this case, this hint is given. Compiling 
with \var{-vd} might result in more information why
 % the directive inline is ignored.
+cg_w_case_incomplete=06059_W_Case statement has unhandled operand values
+% The case statement does not contain labels for all possible values of the 
operand, but not else statement is used.
+%
 %
 % \end{description}
 # EndOfTeX
diff --git a/compiler/msgidx.inc b/compiler/msgidx.inc
index 7100a9eee8..47752cfeb4 100644
--- a/compiler/msgidx.inc
+++ b/compiler/msgidx.inc
@@ -696,6 +696,7 @@ const
   cg_e_function_not_support_by_selected_instruction_set=06056;
   cg_f_max_units_reached=06057;
   cg_n_no_inline=06058;
+  cg_w_case_incomplete=06059;
   asmr_d_start_reading=07000;
   asmr_d_finish_reading=07001;
   asmr_e_none_label_contain_at=07002;
@@ -1106,9 +1107,9 @@ const
   option_info=11024;
   option_help_pages=11025;
 
-  MsgTxtSize = 82706;
+  MsgTxtSize = 82758;
 
   MsgIdxMax : array[1..20] of longint=(
-28,106,349,126,99,59,142,34,221,67,
+28,106,349,126,99,60,142,34,221,67,
 62,20,30,1,1,1,1,1,1,1
   );
diff --git a/compiler/msgtxt.inc b/compiler/msgtxt.inc
index baf2592f67..7ee50d0a45 100644
--- a/compiler/msgtxt.inc
+++ b/compiler/msgtxt.inc
@@ -869,493 +869,500 @@ const msgtxt : array[0..000344,1..240] of char=(
   'ion set: $1'#000+
   '06057_F_Maximum number of units ($1) reached for the current target'#000+
   '06058_N_Call to subroutine "$1" marked as inline is not inlined'#000+
-  

Re: [fpc-devel] [Patch/RFC] Warnings for (in/over)complete case statements

[Martok]

Am 02.01.2019 um 11:06 schrieb Bart:
> So now I will have to add a useless else statement for every case
> statement that uses e.g. integers,
Well, as Jonas argues on fpc-pascal, you are supposed to add useless statements
anyway, which the compiler will then 100% remove...

j/k.

In all seriousness though, that is why I didn't include all integers. But enums
and small ordinals are often used as a "what is this data" tag, where it is
usually an oversight if some value is not checked.
I'm already not super happy this also applies to Char (a OS_8 ordinal), because
it makes case-based regular language parsing quite noisy.


-- 
Regards,
Martok


___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] [Patch/RFC] Warnings for (in/over)complete case statements

[Martok]

Am 02.01.2019 um 11:19 schrieb Michael Van Canneyt:

> Consider the following:
> 
> Type
>TMyClass = class
>Private
>  function GetString(AIndex: Integer): string;
>Published
>  Property MyString : String Index 1 Read GetString;
>end;
> 
> function TMyClass.GetString(AIndex: Integer): string;
> 
> begin
>case AIndex of
> 1 : Result:=GenerateSomestringvalue;
>end;
> end;
> 
> I don't think there should be errors or warnings.

Good example.

Although... you *could* call GetString from some other class member function,
and at that point Result would be undefined. I don't think the compiler can
prove that?
Wait - why does that code not raise a 5033 "Function Result does not seem to be
set"? Add a second property "Property MyOtherString : String Index 2 Read
GetString;", and now its provably wrong. No warning. This seems wrong,
considering what we just talked about on fpc-pascal.

-- 
Regards,
Martok


___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] [Patch/RFC] Warnings for (in/over)complete case statements

[Martok]

Am 02.01.2019 um 11:30 schrieb Florian Klämpfl:
> The compiler is for speed reasons not compiled with $R+/$O+, so the safety 
> else clause have its purpose, but they should
> throw an internal error instead.
And here we are at that other issue again. There is absolutely no guarantee the
safety clause is even present in the compiled ppc. The improved jumptable
generation actually makes it less likely! (the state of $R,$O doesn't change 
this)

So there's really 2 possible goals:
- safety against programmer error: a new stringtype is invented, and not
considered here. The coverage warning will tell the programmer, just as an IE
(maybe) would during testing -- it's a bug either way.
- safety against data error: def deserialized from damaged ppu? Undefined
behaviour, the else clause may not be present in the compiler binary, since all
legal values are handled. Which is exactly what the Unreachable Code warning
would tell you.


This segment of the compiler is specifically the kind of scenario that these
warnings are designed to catch, and also the reason why they are warnings, not
hints. Code that someone wrote and language spec do not agree, and the compiler
will do something that is *very likely* unintended.


-- 
Regards,
Martok


___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] [Patch/RFC] Warnings for (in/over)complete case statements

[Martok]

Am 02.01.2019 um 17:56 schrieb Michael Van Canneyt:
> The compiler can prove that since the methods are private.
> It can see no explicit calls are generated for it.

I thought so too, but even with added explicit calls, there is no warning.
Plus, it could only reason for this class if it was strict private, otherwise,
anyone in this module could access it.

Doesn't seem to be class related, even this very obvioulsy wrong segment
generates no warning:
function GetString(AIndex: Integer): string;
begin
   case AIndex of
1 : Result:= 'abc';
   end;
end;

begin
  writeln(GetString(42));
    end.

-- 
Regards,
Martok


___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] Sorry for poor testing

[Martok]

Am 14.01.2019 um 15:01 schrieb J. Gareth Moreton:
> Martok mentioned doing some checks differently in the bug report in question,
> such as 6 comparisons being faster than a jump table.  Are there any others
> worth mentioning?
Not neccessarily faster, but in that code definitely smaller. Is there a way to
directly estimate how large the generated code might be, maybe something like
"numlabels*constant"?

For the "faster" question, the cache line occupation matters more than
instruction throughput. If your jumptable gets large enough to force a full data
cache line flush, you could have done a lot of instructions in the time the CPU
waits for the data. And I haven't found reliable information how the
Spectre-related microcode updates changed that, given that both cmp/jmp and
jmp[indirect] used to be heavily optimized, but were both affected.


-- 
Regards,
Martok


___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] Fppkg

[Martok]

Am 04.02.2019 um 10:43 schrieb Alfred:
> Would it be possible to add (feature request) a local fppkg.cfg (like fpc.cgf)
> that contains these defaults paths, so they can be modified and used by
> the fppkg executable ? To distinguish between different installs.

May I add that ideally the Lazarus packager-fppkg interface would use the
fppkg.cfg local to $(CompPath). With a proper setup, one could make sure fpc.cfg
and fppkg-config do match - currently they will only agree for the compiler of
the first-ever fppkg.exe invocation on a machine (the one that creates
%LocalAppdata%\Freepascal\fppkg\config), hence break any time the compiler path
changes.

-- 
Regards,
Martok


___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] Fppkg

[Martok]

> Do people actually use fppkg for anything?
Well, *I* dont't know anyone, and the recent stuffs certainly isn't any form of
incentive.

But apparently, fppkg is required to discover fpmake packages that fpc already
knows about, because... reasons?


(Disclaimer: I detest any form of package manager, so my opinions very much
*should* be void.)

-- 
Regards,
Martok


___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] Wrong floating point Currency calculations

[Martok]

> Is this a known bug?
> 
> (Btw. I tested on win32.)

Maybe this?
https://bugs.freepascal.org/view.php?id=33963

-- 
Regards,
Martok

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] TRegistry and Unicode

[Martok]

Am 26.02.2019 um 19:12 schrieb Bart:
> This leaves my initial "itch": input strings are CP_ACP (so can be
> anything), output strings are CP_UTF8 always.
Unless your DefaultSystemCodePage is CP_UTF8 (in which case UTF8Encode is not
needed), the next action on the string will convert away from UTF8 anyways.

String codepages are not stable, there is almost never a point in explicitly
setting one...

-- 
Regards,
Martok


___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Re: [fpc-devel] Successful implementation of inline support forpure assembler routines on x86

[Martok]

Am 17.03.2019 um 18:57 schrieb Florian Klämpfl:
> How is it better than intrinsics support (similiar to gcc/icc etc.)?
It *exists*?

Remember how long it took to get PopCnt support? How about the rest of the BMI?
TBM? AES-NI? Newer AVX?

Implementation of these would be a lot easier if they could be written in code,
even with native-pascal implementations as drop-ins for platforms that don't
support them (or use other mnemonics). Those would just be normal declarations
(in the RTL or in a package), and not require modifying the compiler for all
platforms.
It would also be easier to maintain, as there wouldn't be the need for new
inlinennodes or entirely different node types.

-- 
Regards,
Martok


___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel