from:"Benito van der Zander via fpc\-pascal"

Re: [fpc-pascal] LLVM crash

2023-08-15 Thread Benito van der Zander via fpc-pascal


Hi
On which platform? When I compile the attached tt.pp file with -gw4 
-Clfsanitize=address (LLVM 13, Debian 11, x86-64) and then run it, I 
get the output in tt.txt. It includes line information.


You could try lldb instead of gdb, although gdb should also be able to 
handle debug information generated by LLVM. 


Ubuntu 22.04, LLVM 13

Nothing helps.


only some units are affected

Actually, now I see the warnings during compilation

Assembling xquery
mismatched subprogram between llvm.dbg.addr variable and !dbg attachment
inlinable function call in a function with debug info must have a !dbg 
location
  invoke void 
@"\01XQUERY$_$IXQVALUE_$__$$_$finalize$IXQVALUE"(%typ.XQUERY.IXQValue* 
%reg.1_200)

  to label %.Lj9768 unwind label %.Lj9743
.
 call void @llvm.dbg.addr(metadata %typ.SYSTEM.TRTLCriticalSection* 
%tmp.1, metadata !48637, metadata !DIExpression()), !dbg !48773

label %0
void ()* @"\01XQUERY_$$_finalize$"
!48637 = !DILocalVariable(name: "_zero_$SYSTEM_$$_TRTLCRITICALSECTION", 
scope: !48635, file: !3, line: 10567, type: !1260)
!48635 = distinct !DISubprogram(name: "XQUERY_$$_init$", scope: !3, 
file: !3, line: 3506, type: !7606, scopeLine: 10504, spFlags: 
DISPFlagDefinition, unit: !2)

!48773 = !DILocation(line: 10570, column: 1, scope: !48772)
!48772 = distinct !DISubprogram(name: "XQUERY_$$_finalize$", scope: !3, 
file: !3, line: 10570, type: !7606, scopeLine: 10570, spFlags: 
DISPFlagDefinition, unit: !2)
warning: ignoring invalid debug info in 
/home/theo/lib/fpc/x86_64-linux/xquery.ll




The first appears to be caused by my managed operator patch

Then there is the default issue 
https://gitlab.com/freepascal.org/fpc/source/-/issues/40395

(and you have fixed it while I was still writing this mail)

Then this:

https://gitlab.com/freepascal.org/fpc/source/-/issues/40280 is causing a 
stack corruption


https://gitlab.com/freepascal.org/fpc/source/-/issues/40392 is causing a 
heap corruption



Cheers,
Benito
On 11.08.23 12:57, Jonas Maebe via fpc-pascal wrote:

On 10/08/2023 23:27, Benito van der Zander via fpc-pascal wrote:

i tried to run my program under LLVM (from july fpc)  and it crashes?

Program received signal SIGSEGV, Segmentation fault.
0x0042e5f1in SYSTEM_$$_SYSGETMEM_FIXED$QWORD$$POINTER()
(gdb) bt
#0 0x0042e5f1in SYSTEM_$$_SYSGETMEM_FIXED$QWORD$$POINTER()
#1 0x0041b92ain fpc_ansistr_setlength()
#2 0x00558d52in RESETBUFFER(ABUFFER=0x7fffd560, 
BASECAPACITY=130) at bbutils.pas:1650
#3 INIT(ABUFFER=0x7fffd560, BASECAPACITY=130, AENCODING=65001) at 
bbutils.pas:1639
#4 STRDECODEHTMLENTITIES(result=0x0, P=, L=130, 
ENCODING=65001, FLAGS=...) at bbutils.pas:5527


anyone has seen sysgetmem crash before?


It suggests heap corruption.


Perhaps that is exactly the kind of things ASAN was supposed to detect.


Possibly, yes.

But with ASAN, I get an error somewhere entirely else. And I do not 
understand it, because the function is shown as ~ 5000 lines of 
assembly.


How can I see the mixed code with disassemble /rm in gdb? I tried to 
call fpc -gl, -gs and -gw, and nothing helps


On which platform? When I compile the attached tt.pp file with -gw4 
-Clfsanitize=address (LLVM 13, Debian 11, x86-64) and then run it, I 
get the output in tt.txt. It includes line information.


You could try lldb instead of gdb, although gdb should also be able to 
handle debug information generated by LLVM.



And there are a lot of weird ASAN calls for trivial movs. Like:

0x006f577c<+22204>: 48 8b bb c8 00 00 00 
movrdi,QWORDPTR[rbx+0xc8]
0x006f5783<+22211>: e8 18 cc d0 ff 
call0x4023a0<__asan_report_load8@plt>
0x006f5788<+22216>: e8 13 cc d0 ff 
call0x4023a0<__asan_report_load8@plt>
0x006f578d<+1>: e8 0e cc d0 ff 
call0x4023a0<__asan_report_load8@plt>
0x006f5792<+6>: e8 09 cc d0 ff 
call0x4023a0<__asan_report_load8@plt>

0x006f5797<+22231>: 48 89 c7 movrdi,rax
0x006f579a<+22234>: e8 01 cc d0 ff 
call0x4023a0<__asan_report_load8@plt>

0x006f579f<+22239>: 48 89 cf movrdi,rcx
0x006f57a2<+22242>: e8 09 ca d0 ff 
call0x4021b0<__asan_report_store8@plt>


Are they supposed to be there?


These are generated by LLVM's own code generator, so yes.


Jonas

___
fpc-pascal maillist  -fpc-pascal@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

Re: [fpc-pascal] LLVM LTO

2023-08-15 Thread Benito van der Zander via fpc-pascal


Hi

-Clflto -Clflto-no-system -k-fuse-ld=gold -k-plugin=/path/to/LLVMgold.so 


that helped, but I have to do it without -Clflto-no-system, or it gives 
an error


Then I could run my unit tests:

FPC alone: 25 seconds
LLVM: 21 seconds
LLVM LTO: 20 seconds

does not seem to matter much

Cheers,
Benito
On 11.08.23 12:46, Jonas Maebe via fpc-pascal wrote:

On 10/08/2023 17:06, Benito van der Zander via fpc-pascal wrote:

I tried to use the LLVM Link-Time-Optimization, -Clflto


I should update the wiki page. Under Debian 11, I can tell FPC to use 
gold with lto by specifying the following command line parameters: 
-Clflto -Clflto-no-system -k-fuse-ld=gold -k-plugin=/path/to/LLVMgold.so


Note that compiling the FPC RTL with LTO currently doesn't work under 
Linux, because LLVM does not rename local labels in pure assembler 
procedures when doing so, resulting in duplicate labels.


It works fine when compiling FPC and its units normally, and then only 
using LTO for your program.



Jonas
___
fpc-pascal maillist  - fpc-pascal@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

[fpc-pascal] LLVM LTO

2023-08-11 Thread Benito van der Zander via fpc-pascal


Hallo,

I tried to use the LLVM Link-Time-Optimization, -Clflto

I compiled/installed fpc llvm with a separate prefix, which was already 
difficult to set up


Then the default linker did not want to link it.

The wiki says to use the gold linker. I do not know how to change the 
linker in fpc.cfg. If I use -XP, it changes the prefix. But the gold 
linker is named ld.gold, so one would need an option to change the suffix.


But I called it directly with ld.gold -T link43117.res.

It says

>ld.gold: error: link43117.res:372:8: syntax error, unexpected STRING


I deleted the last line INSERT AFTER .data;. Then it says

>ld.gold: internal error in read_script_file, at ../../gold/script.cc:1638


around here: 
https://sourceware.org/git/?p=binutils-gdb.git;a=blame;f=gold/script.cc;h=d30a4d73991f083f3a1459db06ded4234959fe91;hb=HEAD#l1638



So I gave up on that and tried the llvm linker. ld.lld -T link43117.res


It links, but when I start the program, it gives

Starting program: /home/benito/tmp/a.out

Program received signal SIGSEGV, Segmentation fault.
0xin ??()
(gdb) bt
#0 0xin ??()
#1 0xin ??()




Viele Grüße,

Benito

SEARCH_DIR("/lib/x86_64-linux-gnu/")
SEARCH_DIR("/usr/lib/x86_64-linux-gnu/")
SEARCH_DIR("/lib64/")
SEARCH_DIR("/lib/")
SEARCH_DIR("/usr/lib64/")
SEARCH_DIR("/usr/lib/")
SEARCH_DIR("/usr/lib/thunderbird-addons/")
SEARCH_DIR("/usr/lib/modules/")
SEARCH_DIR("/usr/lib/maxima/")
SEARCH_DIR("/usr/lib/aspell/")
SEARCH_DIR("/usr/lib/gnome-shell/")
SEARCH_DIR("/usr/lib/binfmt-support/")
SEARCH_DIR("/usr/lib/udev/")
SEARCH_DIR("/usr/lib/gcc/")
SEARCH_DIR("/usr/lib/cups/")
SEARCH_DIR("/usr/lib/tmpfiles.d/")
SEARCH_DIR("/usr/lib/firewalld/")
SEARCH_DIR("/usr/lib/java-wrappers/")
SEARCH_DIR("/usr/lib/cryptsetup/")
SEARCH_DIR("/usr/lib/X11/")
SEARCH_DIR("/usr/lib/llvm-14/")
SEARCH_DIR("/usr/lib/cmake/")
SEARCH_DIR("/usr/lib/realmd/")
SEARCH_DIR("/usr/lib/klibc/")
SEARCH_DIR("/usr/lib/pulse-15.99.1+dfsg1/")
SEARCH_DIR("/usr/lib/sasl2/")
SEARCH_DIR("/usr/lib/git-core/")
SEARCH_DIR("/usr/lib/llvm-15/")
SEARCH_DIR("/usr/lib/girepository-1.0/")
SEARCH_DIR("/usr/lib/brltty/")
SEARCH_DIR("/usr/lib/ubiquity/")
SEARCH_DIR("/usr/lib/kernel/")
SEARCH_DIR("/usr/lib/tc/")
SEARCH_DIR("/usr/lib/linux-sound-base/")
SEARCH_DIR("/usr/lib/gcc-cross/")
SEARCH_DIR("/usr/lib/qt5/")
SEARCH_DIR("/usr/lib/bfd-plugins/")
SEARCH_DIR("/usr/lib/crda/")
SEARCH_DIR("/usr/lib/gnome-settings-daemon-42/")
SEARCH_DIR("/usr/lib/GNUstep/")
SEARCH_DIR("/usr/lib/update-notifier/")
SEARCH_DIR("/usr/lib/sysctl.d/")
SEARCH_DIR("/usr/lib/dbus-1.0/")
SEARCH_DIR("/usr/lib/ssl/")
SEARCH_DIR("/usr/lib/pam.d/")
SEARCH_DIR("/usr/lib/netplan/")
SEARCH_DIR("/usr/lib/recovery-mode/")
SEARCH_DIR("/usr/lib/valgrind/")
SEARCH_DIR("/usr/lib/lsb/")
SEARCH_DIR("/usr/lib/man-db/")
SEARCH_DIR("/usr/lib/p7zip/")
SEARCH_DIR("/usr/lib/gold-ld/")
SEARCH_DIR("/usr/lib/pppd/")
SEARCH_DIR("/usr/lib/gnome-settings-daemon-3.0/")
SEARCH_DIR("/usr/lib/hdparm/")
SEARCH_DIR("/usr/lib/binfmt.d/")
SEARCH_DIR("/usr/lib/ispell/")
SEARCH_DIR("/usr/lib/arm-none-eabi/")
SEARCH_DIR("/usr/lib/console-setup/")
SEARCH_DIR("/usr/lib/qemu/")
SEARCH_DIR("/usr/lib/rustlib/")
SEARCH_DIR("/usr/lib/dkms/")
SEARCH_DIR("/usr/lib/os-prober/")
SEARCH_DIR("/usr/lib/rhythmbox/")
SEARCH_DIR("/usr/lib/wine/")
SEARCH_DIR("/usr/lib/usrmerge/")
SEARCH_DIR("/usr/lib/openssh/")
SEARCH_DIR("/usr/lib/python2.7/")
SEARCH_DIR("/usr/lib/init/")
SEARCH_DIR("/usr/lib/locale/")
SEARCH_DIR("/usr/lib/python3.11/")
SEARCH_DIR("/usr/lib/pm-utils/")
SEARCH_DIR("/usr/lib/xserver-xorg-video-intel/")
SEARCH_DIR("/usr/lib/ipxe/")
SEARCH_DIR("/usr/lib/texinfo/")
SEARCH_DIR("/usr/lib/jellyfish1/")
SEARCH_DIR("/usr/lib/memtest86+/")
SEARCH_DIR("/usr/lib/gnupg/")
SEARCH_DIR("/usr/lib/llvm-9/")
SEARCH_DIR("/usr/lib/dpkg/")
SEARCH_DIR("/usr/lib/debug/")
SEARCH_DIR("/usr/lib/mozilla/")
SEARCH_DIR("/usr/lib/gnuplot/")
SEARCH_DIR("/usr/lib/llvm-12/")
SEARCH_DIR("/usr/lib/nvidia/")
SEARCH_DIR("/usr/lib/modules-load.d/")
SEARCH_DIR("/usr/lib/ubuntu-release-upgrader/")
SEARCH_DIR("/usr/lib/linux-boot-probes/")
SEARCH_DIR("/usr/lib/evolution-data-server/")
SEARCH_DIR("/usr/lib/firefox-addons/")
SEARCH_DIR("/usr/lib/software-properties/")
SEARCH_DIR("/usr/lib/environment.d/")
SEARCH_DIR("/usr/lib/openvpn/")
SEARCH_DIR("/usr/lib/systemd/")
SEARCH_DIR("/usr/lib/pkgconfig/")
SEARCH_DIR("/usr/lib/apparmor/")
SEARCH_DIR("/usr/lib/snapd/")
SEARCH_DIR("/usr/lib/xorg/")
SEARCH_DIR("/usr/lib/gnome-session/")
SEARCH_DIR("/usr/lib/ruby/")
SEARCH_DIR("/usr/lib/initramfs-tools/")
SEARCH_DIR("/usr/lib/python3.10/")
SEARCH_DIR("/usr/lib/GraphicsMagick-1.3.38/")
SEARCH_DIR("/usr/lib/pcmciautils/")
SEARCH_DIR("/usr/lib/apache2/")
SEARCH_DIR("/usr/lib/speech-dispatcher-modules/")
SEARCH_DIR("/usr/lib/lp_solve/")
SEARCH_DIR("/usr/lib/geeqie/")
SEARCH_DIR("/usr/lib/encfs/")
SEARCH_DIR("/usr/lib/libreoffice/")
SEARCH_DIR("/usr/lib/grub-legacy/")
SEARCH_DIR("/usr/lib/dh-elpa/")

[fpc-pascal] LLVM crash

2023-08-10 Thread Benito van der Zander via fpc-pascal


Hallo,

i tried to run my program under LLVM (from july fpc)  and it crashes?

Program received signal SIGSEGV, Segmentation fault.
0x0042e5f1in SYSTEM_$$_SYSGETMEM_FIXED$QWORD$$POINTER()
(gdb) bt
#0 0x0042e5f1in SYSTEM_$$_SYSGETMEM_FIXED$QWORD$$POINTER()
#1 0x0041b92ain fpc_ansistr_setlength()
#2 0x00558d52in RESETBUFFER(ABUFFER=0x7fffd560, 
BASECAPACITY=130) at bbutils.pas:1650
#3 INIT(ABUFFER=0x7fffd560, BASECAPACITY=130, AENCODING=65001) at 
bbutils.pas:1639
#4 STRDECODEHTMLENTITIES(result=0x0, P=, L=130, 
ENCODING=65001, FLAGS=...) at bbutils.pas:5527



anyone has seen sysgetmem crash before?


Perhaps that is exactly the kind of things ASAN was supposed to detect.

But with ASAN, I get an error somewhere entirely else. And I do not 
understand it, because the function is shown as ~ 5000 lines of assembly.


How can I see the mixed code with disassemble /rm in gdb? I tried to 
call fpc -gl, -gs and -gw, and nothing helps



And there are a lot of weird ASAN calls for trivial movs. Like:

0x006f577c<+22204>: 48 8b bb c8 00 00 00 movrdi,QWORDPTR[rbx+0xc8]
0x006f5783<+22211>: e8 18 cc d0 ff 
call0x4023a0<__asan_report_load8@plt>
0x006f5788<+22216>: e8 13 cc d0 ff 
call0x4023a0<__asan_report_load8@plt>
0x006f578d<+1>: e8 0e cc d0 ff 
call0x4023a0<__asan_report_load8@plt>
0x006f5792<+6>: e8 09 cc d0 ff 
call0x4023a0<__asan_report_load8@plt>

0x006f5797<+22231>: 48 89 c7 movrdi,rax
0x006f579a<+22234>: e8 01 cc d0 ff 
call0x4023a0<__asan_report_load8@plt>

0x006f579f<+22239>: 48 89 cf movrdi,rcx
0x006f57a2<+22242>: e8 09 ca d0 ff 
call0x4021b0<__asan_report_store8@plt>



Are they supposed to be there?


Viele Grüße,

Benito

___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

Re: [fpc-pascal] Freeing memory with exceptions

2023-05-25 Thread Benito van der Zander via fpc-pascal


Hi,



That 99.99% of people does use it, indicates they simply take the 
overhead because
of the advantages that the managed types offer. 


Or they simply do not know about the overhead

Like I was writing all my code on Windows 98, and never noticed any 
overhead, until I started running benchmarks on  Linux.


Bye,
Benito
On 25.05.23 07:58, Michael Van Canneyt via fpc-pascal wrote:



On Thu, 25 May 2023, Hairy Pixels via fpc-pascal wrote:




On May 24, 2023, at 10:11 PM, Sven Barth via fpc-pascal 
 wrote:


You must have $H+ on and those are AnsiStrings? Why is there 
exception handling involved with AnsiString? I guess it needs this 
just in case an exception is thrown somewhere in the call stack?


Because Ansi- and UnicodeString are managed types. *All* managed 
types managed string types, interfaces, Variants, managed records) 
must be finalized correctly even if an exception occurs.




That's a problem with exceptions then, they are baked into the language
and impose a cost on all managed types now even if we use them or 
not. Even disabling the implicit stack frames (forgot what it's 
called) doesn't

get around this right?


Why do you insist it is a problem ?

Simply don't use managed types and don't use exceptions if you don't 
like the
overhead they cause. It's still perfectly possible: avoid the sysutils 
unit

and you're all set. The system unit does not use exceptions.

That 99.99% of people does use it, indicates they simply take the 
overhead because

of the advantages that the managed types offer.

Which is not to say that FPC should not strive to minimize the overhead.
Conceivably there is some gain possible on non-windows platforms.

Michael.
___
fpc-pascal maillist  - fpc-pascal@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

Re: [fpc-pascal] Freeing memory with exceptions

2023-05-24 Thread Benito van der Zander via fpc-pascal


Hi



Then also run FPC/win32 in wine for a real comparison.



Or perhaps against modern C++ on Linux would also be a real comparison


FPC could at least inline fpc_setjmp in fpc_pushexceptaddr to make it 
only one function call.



These kinds of statements are counter-productive.


That is a very productive optimization idea



Cheers,
Benito
On 24.05.23 13:10, Marco van de Voort via fpc-pascal wrote:


On 24-5-2023 13:00, Benito van der Zander via fpc-pascal wrote:


It is weird that your code calls setjmp? Are you using a non Windows 
platform?  Comparisons with Delphi should be done on Windows where 
the exception systems match. Apples to Apples please.


It is FPC on Linux.

And Delphi 4 on Linux (in WINE)


Then also run FPC/win32 in wine for a real comparison.

 Even if it wants to do the Linux nonsense, FPC could at least inline 
fpc_setjmp in fpc_pushexceptaddr to make it only one function call.


I regret installing Linux. Everything worked better with Windows 98 
and Delphi 4


These kinds of statements are counter-productive.


___
fpc-pascal maillist  - fpc-pascal@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

Re: [fpc-pascal] Freeing memory with exceptions

2023-05-24 Thread Benito van der Zander via fpc-pascal


Hi,

It is weird that your code calls setjmp? Are you using a non Windows 
platform?  Comparisons with Delphi should be done on Windows where the 
exception systems match. Apples to Apples please.


It is FPC on Linux.

And Delphi 4 on Linux (in WINE)

Even if it wants to do the Linux nonsense, FPC could at least inline 
fpc_setjmp in fpc_pushexceptaddr to make it only one function call.



I regret installing Linux. Everything worked better with Windows 98 and 
Delphi 4



Sincerely,
Benito
On 24.05.23 10:14, Marco van de Voort via fpc-pascal wrote:


On 23-5-2023 12:44, Benito van der Zander via fpc-pascal wrote:

Hi,


Depends on your code. 



I wrote all my code in Delphi 4. From 1998 or so. I do not make new 
projects, only maintain old ones.


Delphi 4 felt much better.

Like take:

procedure test;
var s: string;
begin
  s:= 'abc';
end;



It is weird that your code calls setjmp? Are you using a non Windows 
platform?  Comparisons with Delphi should be done on Windows where the 
exception systems match. Apples to Apples please.



___
fpc-pascal maillist  - fpc-pascal@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

Re: [fpc-pascal] Freeing memory with exceptions

2023-05-22 Thread Benito van der Zander via fpc-pascal


Hi,



The compiler will do this wrapping anyway if you use ansistrings, so the
approach with e.g. a generic record will not cause a lot of overhead 
in most
cases. 


But using strings or anything similar causes a lot of overhead

It is really bad



Bye,
Benito
On 21.05.23 18:03, Michael Van Canneyt via fpc-pascal wrote:



On Sun, 21 May 2023, Hairy Pixels via fpc-pascal wrote:




On May 21, 2023, at 2:47 PM, Michael Van Canneyt via fpc-pascal 
 wrote:


Your example will leak memory in any case, even if there is no 
exception,

since you're not freeing the object anywhere..


doh, dumb example on my behalf.



Assuming the result of A is not used outside of Test, the following 
is the

only solution:

procedure Test;

var
A : TObject;
begin
 A:=TObject.Create;
 Try
   // call some code in other unit which raise an exception
   DoThis;
 finally
   A.Free
 end;
end;

You can try to use interfaces, they will be managed by the compiler.


This is what I was worried about, wrapping all functions or needing 
full ARC on all types. Very risk to opt in to this design I would 
say. I remain not a fan of exceptions. :)


They're used all over the place in the RTL and FCL, so you better take 
them

into account.

The compiler will do this wrapping anyway if you use ansistrings, so the
approach with e.g. a generic record will not cause a lot of overhead 
in most

cases.

Michael.
___
fpc-pascal maillist  - fpc-pascal@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

[fpc-pascal] pointer to char vs pchar

2023-03-24 Thread Benito van der Zander via fpc-pascal


Hallo,

why is a pointer to a char not a pchar (for type helpers)?


program Project1;
{$Mode objfpc}{$H+} {$ModeSwitch typehelpers}
type TPcharHelper = type helper for pchar
  function toString(length: integer): string;
end;

function TPcharHelper.toString(length: integer): string;
begin
  SetString(result, self, length);
end;

var c: char;
begin
  c := 'x';
  writeln((@c).toString(1)); //does not compile
  writeln(pchar(@c).toString(1));
end.

Bye,

Benito

___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

Re: [fpc-pascal] Arguments gets corrupted with anonymous nested function

2022-09-07 Thread Benito van der Zander via fpc-pascal


Hi,

https://gitlab.com/freepascal.org/fpc/source/-/issues/38703 



I'm afraid that with -O4 it is by design.
-O4 enables -OoUNCERTAIN, and the documentation about -OoUNCERTAIN 
notes that:


“If uncertain optimizations are enabled, the CSE algorithm assumes that
— If something is written to a local/global register or a 
procedure/function parameter, this value doesn’t overwrite the value 
to which a pointer points.
— If something is written to memory pointed to by a pointer variable, 
this value doesn’t overwrite the value of a local/global variable or a 
procedure/function parameter.”




is it doing that? did it ever do that? It might break a lot of my code

But I cannot find any place in FPC where it is doing any OoUNCERTAIN 
optimizations


Best,
Benito
On 07.09.22 13:51, Peter B via fpc-pascal wrote:

I'm wondering if this is related to
https://gitlab.com/freepascal.org/fpc/source/-/issues/38703

That caused a parameter corruption in a standard function.
https://lists.freepascal.org/pipermail/fpc-pascal/2022-March/060361.html

Should be easy to check. Just try compiling with -O1


Cheers,
Peter
___
fpc-pascal maillist  - fpc-pascal@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

Re: [fpc-pascal] Compiler flag define or $IFOPT for optimizations

2022-06-03 Thread Benito van der Zander via fpc-pascal


Hi,
However, you can possibly solve it by always taking the options from 
an environment variable to the command line and then including 
contents of this command line to your sources. 


Unfortunately it is picking up all kinds of options from fpc.cfg and 
Lazarus.


Like in this case, I tried to disable range checking, and it is still 
enabled.



> You can introduce something like that yourself by always building all 
your sources from scratch, but then the solution outlined above should 
work for you.


I am setting up a build server that compiles the entire project from 
scratch on every commit.



Best,
Benito
On 28.05.22 14:34, Tomas Hajny via fpc-pascal wrote:

On 2022-05-28 13:44, Benito van der Zander via fpc-pascal wrote:


Hi,


I want to show how my program was compiled.

Now I have string like "FPC3.2.2 i386-Linux R+C+" from

  compiler := 'FPC' + {$INCLUDE %FPCVERSION%} + ' ' + {$INCLUDE
%FPCTargetCPU%}+'-'+{$INCLUDE %FPCTargetOS%}+ ' ' + {$IfOpt
R+}+'R+'{$endif} {$IfOpt S+}+'S+'{$endif} {$IfOpt O+}+'O+'{$endif}
{$IfOpt Q+}+'Q+'{$endif} {$IfOpt M+}+'M+'{$endif} {$IfOpt
C+}+'C+'{$endif};

But the optimization level (-O2 or -O1 ...) is missing.

Is there an IFOPT for that? Or a define with all the arguments


I don't think that there's such an option at the moment. However, you 
can possibly solve it by always taking the options from an environment 
variable to the command line and then including contents of this 
command line to your sources. I don't think that it makes much sense 
for the compiler to provide such an option, because unlike the 
compiler version, different options (including the optimization level) 
may be used for compilation of different units and there's no such a 
thing as a general optimization level valid for the complete compiled 
program from the compiler point of view. You can introduce something 
like that yourself by always building all your sources from scratch, 
but then the solution outlined above should work for you.


Tomas
___
fpc-pascal maillist  - fpc-pascal@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

Re: [fpc-pascal] Feature Announcement: Function References and Anonymous Functions

2022-05-28 Thread Benito van der Zander via fpc-pascal


Hi,


Sort((left, right) begin
 if left < right then
   result := -1
 else if left > right then
   result := 1
 else
   result := 0;
   end);


One could introduce the @ operator to get a reference to a block of code.

Sort( @(left, right) begin
if left < right then
  result := -1
else if left > right then
  result := 1
else
  result := 0;
  end);


The "result" variable is also ugly. You could make "if" an expression:

Sort( @(left, right) begin
result :=
  if left < right then -1
  else if left > right then 1
  else 0;
  end);


Then the begin end could be removed too. Make @:= an operator to turn an 
expression to a function


Sort( @(left, right) :=
   if left < right then -1
   else if left > right then 1
   else 0;
  );





Benito
On 28.05.22 08:47, Hairy Pixels via fpc-pascal wrote:

I’ve had some time to play with this now and my first piece of feedback is that 
given my experience with other languages, the most common usage of closures is 
by passing them as arguments to functions.

Compared to the other languages I’m using now I’d say that we should be 
inferring more of the context of the receiving function type and not requiring 
the programmer to type out the full function header as in the example below 
(especially in the case there are no local variables declared).

Sort(function(left, right: Double): integer
   begin
 if left < right then
   result := -1
 else if left > right then
   result := 1
 else
   result := 0;
   end);

It’s hard to say what the best policy is for Pascal but some combination of the 
function/procedure keyword, parameter type names and return type could be 
omitted or shortened in various ways.

Given we know the function type from the parameter list we could infer most of 
the information and provide a more limited set of syntax, for example like this 
(Swift like):

Sort((left, right) begin
 if left < right then
   result := -1
 else if left > right then
   result := 1
 else
   result := 0;
   end);

There is even the most limited shorthand (from Swift again) which uses token to 
represent the arguments as they are ordered.

Sort(begin
   if $0 < right then
 result := -1
   else if $0 > $1 then
 result := 1
   else
 result := 0;
 end);

Regards,
Ryan Joseph

___
fpc-pascal maillist  -fpc-pascal@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

[fpc-pascal] Compiler flag define or $IFOPT for optimizations

2022-05-28 Thread Benito van der Zander via fpc-pascal


Hallo,


I want to show how my program was compiled.

Now I have string like "FPC3.2.2 i386-Linux R+C+" from


  compiler := 'FPC' + {$INCLUDE %FPCVERSION%} + ' ' + {$INCLUDE 
%FPCTargetCPU%}+'-'+{$INCLUDE %FPCTargetOS%}+ ' ' + {$IfOpt 
R+}+'R+'{$endif} {$IfOpt S+}+'S+'{$endif} {$IfOpt O+}+'O+'{$endif} 
{$IfOpt Q+}+'Q+'{$endif} {$IfOpt M+}+'M+'{$endif} {$IfOpt C+}+'C+'{$endif};


But the optimization level (-O2 or -O1 ...) is missing.

Is there an IFOPT for that? Or a define with all the arguments


Bye,

Benito

___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

Re: [fpc-pascal] StrToInt is using ShortString buffer?

2022-01-14 Thread Benito van der Zander via fpc-pascal


Hi,


I don’t see what the problem with a shortstring buffer is.. strtoint 
could not make use of an input string longer than 256 characters 
anyway, because the output integer could not have more than 19 digits 
anyway.. so an input string longer than this would be useless… just 
convert to a shortstring and to strtoint.




the input might be

Re: [fpc-pascal] StrToInt is using ShortString buffer?

2022-01-12 Thread Benito van der Zander via fpc-pascal


Hi,



If it is correct: Can we skip using ShortString buffer and have e.g. 
'common' PChar base function for AnsiString/ShortString? 


I keep arguing for that as well

You can always create such a function yourself,  gain experience using 
it, and share the source with us.




I wrote that for you:

https://forum.lazarus.freepascal.org/index.php/topic,55472.msg414973.html#msg414973

https://github.com/benibela/bbutils/blob/master/bbutils.pas#L3446-L3601

Cheers,
Benito

___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

Re: [fpc-pascal] Where can I find working code for xidel and dependencies

2022-01-11 Thread Benito van der Zander via fpc-pascal


Hi,


xquery.internals.common.pas(434,80) Fatal: Cannot find PasDblStrUtils used by
xquery.internals.common of package internettools.



that is here: github.com/bero1985/pasdblstrutils

Download src/PasDblStrUtils.pas. Do not clone the repository when you 
have a slow internet connection



It only became a dependency last week. You could look at the last 
internettools commits and undo them



And you will also need FLRE, which has a fork here: github.com/benibela/flre

Linux also requires Synapse


After trying to fix this I decided to search for binaries and found them here:
https://sourceforge.net/projects/videlibri/files/Xidel/Xidel%200.9.8/
These I believe are all for PC style CPU:s


Do not use those, they are 3 years old

The recent builds are: at 
https://sourceforge.net/projects/videlibri/files/Xidel/Xidel%20development/



Best,
Benito
On 09.01.22 18:43, Bo Berglund via fpc-pascal wrote:

On Sun, 09 Jan 2022 17:46:00 +0100, Bo Berglund via fpc-pascal
 wrote:


I have been adviced to use xidel to analyze the content of websites, so when
searcing for it I found that it is a FreePascal utility with the sources on
GitHub.
https://www.videlibri.de/xidel.html
So I tried to get it going.
Buit when I tried to compile the utility using Lazarus 2.0.12/FPC 3.2.0 it
failed to find package "internettools".
After some searching I found it on GitHub:
https://github.com/benibela/internettools

But still no luck, I get errors when trying to compile the internettools
package

UPDATE:
---
After trying to fix this I decided to search for binaries and found them here:
https://sourceforge.net/projects/videlibri/files/Xidel/Xidel%200.9.8/
These I believe are all for PC style CPU:s

So I will not need to build it myself unless I want to run it on a RaspberryPi
of course.

___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

Re: [fpc-pascal] Traits Proposal

2021-02-18 Thread Benito van der Zander via fpc-pascal


Hi,

The problem is that, we like OOP inheritance  but when we extend classes we are 
forced into a single hierarchy.


there is another way to extend classes without inheritance: type helpers

Traits are like reverse type helpers. With the type helper you first 
declare the class and then the extending helper.


=== code begin ===
type
  TTest = class(TObject)
  public
    procedure test(a: integer);
  end;
  TTrait = class helper for TTest
    procedure test2;
  end;

procedure TTest.test(a: integer);
begin
  writeln(a);
end;

procedure TTrait.test2;
begin
  test(10);
end;
//var x: ttest2;
//x.test2

=== code end ===


With traits you could first define the extension, and then the class:

=== code begin ===
type
  TTrait = trait
    procedure test2;
  end;
  TTest = class(TObject, TTrait)
  public
    procedure test(a: integer);
  end;

procedure TTest.test(a: integer);
begin
  writeln(a);
end;

procedure TTrait.test2;
begin
  test(10); //here it needs to know the test function exist. Perhaps 
declare it in the the trait, too?

end;
//var x: ttest2;
//x.test2
=== code end ===

But otherwise, it looks exactly the same

Perhaps the type helper code could be reused for traits





Cheers,
Benito


On 18.02.21 04:37, Ryan Joseph via fpc-pascal wrote:




On Feb 17, 2021, at 9:59 AM, Adriaan van Os via fpc-pascal 
 wrote:

1. multiple inheritance is nice to have, but it has the big issue that the 
inheritance tree becomes an inheritance graph and that makes overrules 
ambiguent.
2. interfaces don't have this issue with multiple inheritance, because they 
just declare, not implement
3. but that is also the weakness of interfaces,  as we don't want to 
reimplement the same code each time
4. so, we really want an multiple-inheritance graph at the declaration level 
with clear tree-like unambigous inheritance paths at the implementation level
5. thus, the idea is to "push-in" implementation code into an interface that 
integrates fully at the declaration level but is independent at the implementation level.

I would say that's right.

The problem is that, we like OOP inheritance  but when we extend classes we are forced 
into a single hierarchy. We could use existing delegation patterns and dot-notation  like 
obj.helper.DoSomething or by breaking out entirely and using plain functions  but then we 
lose some of what makes OOP nice, which is, simply saying 
"something.DoFunction". It may be trivial in terms of typing but it's elegant 
and in my opinion clean code which is not tedious to write makes happy and more 
productive.


Here's a more practical example but there are other possibilities for 
composition patterns (I'l think of examples later). I copied this from the RTL 
and made some changes. Assume you have this hierarchy and you want to add some 
methods/data to JPEG and GIF images but not TIFF and PNG (they have some 
special compression needs or something like that). What we would do now is 
probably just dump them into TCustomBitmap (and bloat TIFF/PNG)  or make 
another subclass just to store the extra methods (which is useless besides 
being a method store for 2 specific other classes).

What I want with traits is that I can extend those 2 classes with specific 
functionality, still retain the object-orientedness of the syntax and not get 
trapped trying to inject stuff into a hierarchy where it doesn't really belong 
anyways.


TTIFFBitmap   TPNGBitmap   TJPEGBitmap   TGIFBitmap
 |
 TCustomBitmap
 |
 TRasterImage
 |
 TGraphic
 |
 TPersistent
 |
 TObject


Regards,
Ryan Joseph

___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

Re: [fpc-pascal] interfaces and smartpointers [was Traits Proposal]

2021-02-17 Thread Benito van der Zander via fpc-pascal



On 17.02.21 21:43, Ryan Joseph via fpc-pascal wrote:

So where is your 10% performance hit coming from then?

on init:

InitInterfacePointers or TInterfacedObject.AfterConstruction?



I benchmarked it years ago, I do not remember the details.

But InitInterfacePointers was bad. Just look at it:

  procedure InitInterfacePointers(objclass: tclass;instance : pointer);

    var
  ovmt: PVmt;
  i: longint;
  intftable: pinterfacetable;
  Res: pinterfaceentry;
    begin
  ovmt := PVmt(objclass);
  while assigned(ovmt) and {$ifdef VER3_0}(ovmt^.vIntfTable <> 
@emptyintf){$else}assigned(ovmt^.vIntfTable){$endif} do

    begin
  intftable:=ovmt^.vIntfTable;
  {$ifdef VER3_0}
  if assigned(intftable) then
  {$endif VER3_0}
  begin
    i:=intftable^.EntryCount;
Res:=@intftable^.Entries[0];
    while i>0 do begin
  if Res^.IType = etStandard then
    ppointer(@(pbyte(instance)[Res^.IOffset]))^:=
  pointer(Res^.VTable);
  inc(Res);
  dec(i);
    end;
  end;
  ovmt:=ovmt^.vParent;
    end;
    end;


But reusing objects avoids calling all the functions from create. (but 
reusing is also expensive, besides the memory increase, it needs a free 
list and thread safe handling of the head of the list)




otherwise, during the wrapper function? As Sven pointed out the ref counting 
happens with records operators also. ARC is general may just be slow if you 
implement it everywhere.



The wrapper function was not included in the 10%


___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

Re: [fpc-pascal] interfaces and smartpointers [was Traits Proposal]

2021-02-17 Thread Benito van der Zander via fpc-pascal


Hi,


1) Implicit cast to ITest which does a runtime lookup (but there's interface 
name so no string comparison like Supports?)


just open the disassembler window in Lazarus and single step through all 
the instructions. Then you see everything


create alone calls a bunch of methods:

TInterfacedObject.NewInstance
TObject.NewInstance
getmem
TObject.InitInstance
fillchar
InitInterfacePointers
fpc_pushexceptaddr (on linux only?)
fpc_setjmp
TInterfacedObject.AfterConstruction


(interfaces with TInterfacedObject are even worse than implementing the 
interface yourself )


The implicit casting is very fast, a single assembly instruction:

add    $0x20,%rsi

But then it also calls fpc_intf_assign for reference counting



2) calling "println" there is a call to a wrapper function (called "thunks")?


Yes. Basically the wrapper function casts the interface back to the 
class (sub    $0x20,%rdi) before calling the actual method. Because the 
class method assumes self is the class and not an interface





Bye,
Benito
On 17.02.21 17:37, Ryan Joseph via fpc-pascal wrote:



On Feb 17, 2021, at 8:27 AM, Benito van der Zander via fpc-pascal 
 wrote:

var c: ITest;
begin
   c := TTest.Create(123);
   c.println;

So this is where you're getting your performance penalties? Correct me if I'm 
wrong but two things happen here:

1) Implicit cast to ITest which does a runtime lookup (but there's interface 
name so no string comparison like Supports?)
2) calling "println" there is a call to a wrapper function (called "thunks")?

Regards,
Ryan Joseph

___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

Re: [fpc-pascal] interfaces and smartpointers [was Traits Proposal]

2021-02-17 Thread Benito van der Zander via fpc-pascal


Hi,



(1) A record is not a pointer. So that would require some implicit 
referencing in the property


(2) If it was managed, it would be an allocation, so I don't 
understand this.



I am talking about replacing interfaces with an record.

For example this

/

type ITest = interface
  procedure println;
end;
TTest = class(TInterfacedObject, ITest)
  v: integer;
  constructor Create(av: integer);
  procedure println;
end;

constructor TTest.Create(av: integer);
begin
  v := av;
end;

procedure TTest.println;
begin
  writeln(v);
end;

var c: ITest;
begin
  c := TTest.Create(123);
  c.println;
/


would become:


/

type

TTestRec = class
  rc: Integer;
  v: integer;
  constructor Create(av: integer);
  procedure println;
end;
RTest = record
  ptr: TTestRec;
  procedure println; inline;
  class operator :=(c: TTestRec): RTest;
  class operator Initialize(var aRec: RTest);
  class operator finalize(var aRec: RTest);
  class operator AddRef(var aRec: RTest);
end;

constructor TTestRec.Create(av: integer);
begin
  v := av;
end;

procedure TTestRec.println;
begin
  writeln(v);
end;

procedure RTest.println;
begin
  ptr.println; //the wrapper function is inlined
end;

class operator RTest.:=(c: TTestRec): RTest;
begin
  result := default(RTest);
  result.ptr := c;
  if c <> nil then
    InterlockedIncrement(c.rc);
end;

class operator RTest.Initialize(var aRec: RTest);
begin
  aRec.ptr := nil;
end;

class operator RTest.finalize(var aRec: RTest);
begin
  if aRec.ptr <> nil then
    if InterlockedDecrement(aRec.ptr.rc) = 0 then
  aRec.ptr.Free;
end;

class operator RTest.AddRef(var aRec: RTest);
begin
  if aRec.ptr <> nil then
    InterlockedIncrement(aRec.ptr.rc);
end;

var
    r: RTest;
begin
  r := TTestRec.Create(123);
  r.println;
/

Or even replace the class with a record, too:


/

type
PTestRec2 = ^TTestRec2;
TTestRec2 = record
  rc: Integer;
  v: integer;
  class function Create(av: integer): PTestRec2; static;
  procedure println;
end;
RTest2 = record
  ptr: ^TTestRec2;
  procedure println; inline;
  class operator :=(c: PTestRec2): RTest2;
  class operator Initialize(var aRec: RTest2);
  class operator finalize(var aRec: RTest2);
  class operator AddRef(var aRec: RTest2);
end;

class function TTestRec2.Create(av: integer): PTestRec2;
begin
  new(result);
  result^.rc := 0;
  result^.v := av;
end;

procedure TTestRec2.println;
begin
  writeln(v);
end;

procedure RTest2.println;
begin
  ptr^.println;
end;

class operator RTest2.:=(c: PTestRec2): RTest2;
begin
  result := default(RTest2);
  result.ptr := c;
  if c <> nil then
    InterlockedIncrement(c^.rc);
end;

class operator RTest2.Initialize(var aRec: RTest2);
begin
  aRec.ptr := nil;
end;

class operator RTest2.finalize(var aRec: RTest2);
begin
  if aRec.ptr <> nil then
    if InterlockedDecrement(aRec.ptr^.rc) = 0 then
  dispose(aRec.ptr);
end;

class operator RTest2.AddRef(var aRec: RTest2);
begin
  if aRec.ptr <> nil then
    InterlockedIncrement(aRec.ptr^.rc);
end;
var
    r2: RTest2;
begin
  r2 := TTestRec2.Create(123);
  r2.println;
/




Not sure if it is actually faster. That needs to be investigated.

But it definitely helps with the memory usage:

  writeln(ttest.InstanceSize);
  writeln(ttestrec.InstanceSize);
  writeln(sizeof(ttestrec2));

40
16
8

With many small objects it should be faster just because it fits better 
in the cache.


Cheers,
Benito
On 17.02.21 14:31, Marco van de Voort via fpc-pascal wrote:


Op 2021-02-17 om 00:02 schreef Benito van der Zander via fpc-pascal:


And there often is a lot of unintentional deep copying. This is also 
why a property returning a record is fairly useless except for 
extremely small records like TPoint (and even that is not optimal no 


But a managed record to replace an interface, would only contain a 
single pointer/class ref. That can be copied fast


(1) A record is not a pointer. So that would require some implicit 
referencing in the property


(2) If it was managed, it would be an allocation, so I don't 
understand this.



___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

Re: [fpc-pascal] Traits Proposal

2021-02-16 Thread Benito van der Zander via fpc-pascal


Hi,


And there often is a lot of unintentional deep copying. This is also 
why a property returning a record is fairly useless except for 
extremely small records like TPoint (and even that is not optimal no 


But a managed record to replace an interface, would only contain a 
single pointer/class ref. That can be copied fast




Bye,
Benito
On 16.02.21 21:49, Marco van de Voort via fpc-pascal wrote:


Op 2021-02-16 om 21:33 schreef Ryan Joseph via fpc-pascal:


On Feb 16, 2021, at 1:27 PM, Marco van de Voort via fpc-pascal 
 wrote:


And there often is a lot of unintentional deep copying. This is also 
why a property returning a record is fairly useless except for 
extremely small records like TPoint (and even that is not optimal no
deep copying? You mean from ref counted types like "array of" and 
ansistring?


No, use of TP objects and records in e.g. properties in not carefully 
crafted code. They are a lot less userfriendly.



___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

Re: [fpc-pascal] Traits Proposal

2021-02-16 Thread Benito van der Zander via fpc-pascal

Hi,

If you need to create 1000 class instances each frame then you have a 
flaw in your logic in my opinion.

I have more like a million class instances

For my XPath stuff, and every returned value is put in a variant-like 
class. Selecting all nodes on an GB large XML, could even create almost 
a billion class instances

I once did a benchmark. It is 10% faster to reuse the class instances 
rather than recreating them, my commit log says. And I only implemented 
one interface (actually, two interfaces, since IUnknown always gets 
pulled in, too. )

Interfaces are not slow because they are are interfaces! When you call 
a interface method it has the same costs as a virtual method call! And 
the cost for reference counting that interfaces have right now would 
be there for ARC as well.

But it is not calling the method, it is calling the wrapper function

That is a additional second method call/jump for every interface method 
call.

E.g. _AddRef on TInterfacedObject as an interface, does not call 
TInterfacedObject._AddRef, it calls this:

WRPR_$SYSTEM_$$_TINTERFACEDOBJECT_$_IUNKNOWN_$_1_$_SYSTEM$_$TINTERFACEDOBJECT_$__$$__ADDREF$$LONGINT 

00424600 4883ef10 sub    $0x10,%rdi
00424604 e997f8feff   jmpq   0x413ea0 

Bye,
Benito
On 16.02.21 19:48, Sven Barth via fpc-pascal wrote:
Ryan Joseph via fpc-pascal > schrieb am Di., 16. Feb. 
2021, 19:21:

>
> There we have:
>
> * slower creation of the class, because each implemented
interface adds another VMT reference to the class which needs to
be initialized.

How bad is this? We need to make a test case we can profile. For
example if you have a game that creates 1000 classes 60 frames per
second. If adding interfaces to the classes hurts this process we
may have a deal breaker.

If you need to create 1000 class instances each frame then you have a 
flaw in your logic in my opinion.

> * slow reference counting. Especially if it is thread safe and
exception safe with the implicit exception block

It's a whole other topic but FPC needs opt-in ARC at the language
level for classes. Interfaces are not only slow but then you need
to pass around a reference to the interface instead of the class
you actually care about. It makes no sense to me whatsoever.

Interfaces are not slow because they are are interfaces! When you call 
a interface method it has the same costs as a virtual method call! And 
the cost for reference counting that interfaces have right now would 
be there for ARC as well.

Regards,
Sven

___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

Re: [fpc-pascal] Traits Proposal

2021-02-16 Thread Benito van der Zander via fpc-pascal





There are no significant performance implications of interfaces. 
They're essentially a virtual method call, something that one is doing 
all day long with Object Pascal classes. 



Interfaces are extremely slow. Virtual method calls are also slow. I 
have been using interfaces for reference counting, and have been 
thinking to replace it all with managed records because they are so slow 
(unfortunately records can also be slow because fpc does not always keep 
them in registers even if they are pointer sized)


There we have:

* slower creation of the class, because each implemented interface adds 
another VMT reference to the class which needs to be initialized.


* slower method calling because of the wrapper function

* extremely slow casting an interface to the class. Never cast an 
interface, better add a method to the interface returning a class reference


* slow reference counting. Especially if it is thread safe and exception 
safe with the implicit exception block



On 14/02/2021 11:31, Sven Barth via fpc-pascal wrote:

Am 14.02.2021 um 01:09 schrieb Ben Grasset via fpc-pascal:
This seems possibly a *little* too similar to the existing Interface 
type in Object Pascal, however, I *would* really like to see some 
kind of functionality that basically amounts to "has the same 
capabilities as Interfaces and works on records and objects too, but 
does NOT require any kind of heap allocation".


My personal idea for this would be to allow for duck typing regarding 
interfaces: as long as the methods of the class, object or record 
satisfy those of the interface then it's possible to assign it. In 
case of objects and records one would mainly use raw interfaces (aka 
corba ones) instead of reference counted one as otherwise one would 
need to provide the methods of IInterface as well though one might not 
be able to implement them in a usable way as stack objects simply 
don't work that way.


So whether it be this, or just an improvement on the Interfaces we 
already have, I'd definitely personally be in favor of something that 
"works like Interfaces except minus the negative performance 
implications."


There are no significant performance implications of interfaces. 
They're essentially a virtual method call, something that one is doing 
all day long with Object Pascal classes.


Regards,
Sven
___
fpc-pascal maillist  - fpc-pascal@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

Re: [fpc-pascal] Question about System.Move()

2021-01-29 Thread Benito van der Zander via fpc-pascal


Hi,


Should "A" have a reference count of 1, and it is assigned a new value 
on another thread, its reference count will decrease to zero and the 
string and its descriptor are both freed. The value of "A" will be set 
to nil. 



If the reference count is 1, there is no other thread involved.

If there was another thread, there would be a reference in this thread 
and in the other thread, so the reference count would be at least 2



It is exactly as thread safe as the unoptimized function. If the 
reference count was 1, the unoptimized function would not work either, 
since the thread swap might occur just before the first instruction, and 
the strings are freed before the function can do anyhting



Bye,
Benito
On 28.01.21 08:08, Derek Edson via fpc-pascal wrote:
Your simplified code would not be thread safe. A thread swap after the 
first instruction of


        mov (%rdi), %rax

could potentially cause a problem. RAX (A) contains the pointer to the 
string descriptor, which includes the pointer to the actual string 
data and the reference count.


Should "A" have a reference count of 1, and it is assigned a new value 
on another thread, its reference count will decrease to zero and the 
string and its descriptor are both freed. The value of "A" will be set 
to nil.


RAX will not have been updated, so after

        mov %rax, (%rsi)

"B" will be pointing to an invalid location. Its content is now 
dependent on memory reallocation.


In this situation, changing "B" after the return from stringSwap 
procedure might again cause the string to be freed (assuming the 
memory was not reallocated), causing a double free error.


Use of the internal procedures to increment the string reference count 
of both strings before your code and then decrementing the reference 
counts afterwards would probably work.


The same probably applies to other reference count objects.

Derek

On Thu, Jan 28, 2021 at 11:35 AM Benito van der Zander via fpc-pascal 
<mailto:fpc-pascal@lists.freepascal.org>> wrote:


Hi,




Procedure ManagedMove(const source: T;var dest: T;count:
SizeInt);


In principle a good idea. However this is one of those cases
where you'd definitely need to use constref instead of const.



Or var, since the source might be cleared


And perhaps there could be a special attribute to mark which
kind of moving is needed, e.g..

  type [moveable] TA = record
  type [referencecounted] TA = record
  type [nonmoveable] TA = record


No, thank you.


But it could help a lot with optimizations. For moveable types it
could omit calling an assignment function and just do a memory
copy; and for refcounted tyes it could that when the number of
references does not change.



Like a simple swap function:

procedure stringSwap(var a, b: string);
var t: string;
begin
  t := a;
  a := b;
  b := t;
end;

A smart compiler could detect that it is a refcounted type and the
number of references stays the same, and thus optimize it into:

    mov    (%rdi),%rax
    mov    (%rsi),%rdx
    mov    %rdx,(%rdi)
    mov    %rax,(%rsi)
    retq

But FPC turns it into:

    begin
    push   %rbx
    push   %r12
    lea    -0x68(%rsp),%rsp
    mov    %rdi,%rbx
    mov    %rsi,%r12
    movq   $0x0,(%rsp)
    lea    0x8(%rsp),%rdx
    lea    0x20(%rsp),%rsi
    mov    $0x1,%edi
    callq  0x4324c0 
    mov    %rax,%rdi
    callq  0x41fba0 
    movslq %eax,%rdx
    mov    %rdx,0x60(%rsp)
    test   %eax,%eax
    jne    0x469191 
    t := a;
    mov    (%rbx),%rsi
    mov    %rsp,%rdi
    callq  0x428d00 
    a := b;
    mov    (%r12),%rsi
    mov    %rbx,%rdi
    callq  0x428d00 
    b := t;
    mov    (%rsp),%rsi
    mov    %r12,%rdi
    callq  0x428d00 
    callq  0x432830 
    end;
    mov    %rsp,%rdi
    callq  0x428ca0 
    mov    0x60(%rsp),%rax
    test   %rax,%rax
    je 0x4691b8 
    callq  0x4329e0 
    movq   $0x0,0x60(%rsp)
    jmp    0x469191 
    lea    0x68(%rsp),%rsp
    pop    %r12
    pop    %rbx
    retq




Then you want to simply use ismanagedtype and move(). Moving the
move() to a separate generic procedure will only lead to many
instantiations of what is basically a move() procedure.


The goal is to reduce instantiations.
If there are n generic (collection) classes, C1, C2, .., Cn, and k
types used in the collections, T1, T2, .. Tk, there are n*k
collection class instantiation C1, C1, .. C1, C2,
... Cn
If each collection class does its own Finalize-Loop/Move/Fillchar
there are also n*k of these move implementations. So like 3*n*k calls.
When any collection can call the same Manag

Re: [fpc-pascal] Question about System.Move()

2021-01-27 Thread Benito van der Zander via fpc-pascal


Hi,




Procedure ManagedMove(const source: T;var dest: T;count: SizeInt);


In principle a good idea. However this is one of those cases where 
you'd definitely need to use constref instead of const.



Or var, since the source might be cleared


And perhaps there could be a special attribute to mark which kind
of moving is needed, e.g..

  type [moveable] TA = record
  type [referencecounted] TA = record
  type [nonmoveable] TA = record


No, thank you.

But it could help a lot with optimizations. For moveable types it could 
omit calling an assignment function and just do a memory copy; and for 
refcounted tyes it could that when the number of references does not 
change.




Like a simple swap function:

procedure stringSwap(var a, b: string);
var t: string;
begin
  t := a;
  a := b;
  b := t;
end;

A smart compiler could detect that it is a refcounted type and the 
number of references stays the same, and thus optimize it into:


    mov    (%rdi),%rax
    mov    (%rsi),%rdx
    mov    %rdx,(%rdi)
    mov    %rax,(%rsi)
    retq

But FPC turns it into:

    begin
    push   %rbx
    push   %r12
    lea    -0x68(%rsp),%rsp
    mov    %rdi,%rbx
    mov    %rsi,%r12
    movq   $0x0,(%rsp)
    lea    0x8(%rsp),%rdx
    lea    0x20(%rsp),%rsi
    mov    $0x1,%edi
    callq  0x4324c0 
    mov    %rax,%rdi
    callq  0x41fba0 
    movslq %eax,%rdx
    mov    %rdx,0x60(%rsp)
    test   %eax,%eax
    jne    0x469191 
    t := a;
    mov    (%rbx),%rsi
    mov    %rsp,%rdi
    callq  0x428d00 
    a := b;
    mov    (%r12),%rsi
    mov    %rbx,%rdi
    callq  0x428d00 
    b := t;
    mov    (%rsp),%rsi
    mov    %r12,%rdi
    callq  0x428d00 
    callq  0x432830 
    end;
    mov    %rsp,%rdi
    callq  0x428ca0 
    mov    0x60(%rsp),%rax
    test   %rax,%rax
    je 0x4691b8 
    callq  0x4329e0 
    movq   $0x0,0x60(%rsp)
    jmp    0x469191 
    lea    0x68(%rsp),%rsp
    pop    %r12
    pop    %rbx
    retq




Then you want to simply use ismanagedtype and move().  Moving the 
move() to a separate generic procedure will only lead to many 
instantiations of what is basically a move() procedure.


The goal is to reduce instantiations.
If there are n generic (collection) classes, C1, C2, .., Cn, and k types 
used in the collections, T1, T2, .. Tk, there are n*k collection class 
instantiation C1, C1, .. C1, C2, ... Cn
If each collection class does its own Finalize-Loop/Move/Fillchar there 
are also n*k of these move implementations. So like 3*n*k calls.
When any collection can call the same ManagedMove, there should only be 
k ManagedMove instantiations and n*k calls to ManagedMove, which is only 
one call. So it is k + n*k calls,  which is around a third of 3*n*k


Bye,
Benito
On 11.01.21 18:51, Sven Barth via fpc-pascal wrote:
Benito van der Zander via fpc-pascal <mailto:fpc-pascal@lists.freepascal.org>> schrieb am Mo., 11. Jan. 
2021, 15:26:


Hi,

perhaps a  safe, generic function for this copying could be added
to the RTL. Like:

Procedure ManagedMove(const source: T;var dest: T;count: SizeInt);


In principle a good idea. However this is one of those cases where 
you'd definitely need to use constref instead of const.


And when you use IsManagedType, it does not distinguish standard
strings with such weird managed types.


You can additionally use GetTypeKind as well. Unlike TypeInfo it 
directly returns the TTypeKind (which for this case is enough) and is 
considered constant.


And perhaps there could be a special attribute to mark which kind
of moving is needed, e.g..

  type [moveable] TA = record
  type [referencecounted] TA = record
  type [nonmoveable] TA = record


No, thank you.

Regards,
Sven

___
fpc-pascal maillist  -fpc-pascal@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

Re: [fpc-pascal] Question about System.Move()

2021-01-11 Thread Benito van der Zander via fpc-pascal


Hi,

perhaps a  safe, generic function for this copying could be added to the 
RTL. Like:


Procedure ManagedMove(const source: T;var dest: T;count: SizeInt);


a) For non-managed types it would be the same as Move(source, dest, 
count*sizeof(T))


b) For reference counted types (like strings or interfaces or records of 
them), it would do:


1. finalize dest
2. normal Move
3. clear source, with fillchar

(for ranges where source and dest overlap, it should skip 1 and 3 for 
the overlapping elements. Like when you shift some elements in an arrays 
by two elements to the left, it only has to finalize the two overridden 
elements before the shifted elements, and clear the last two shifted 
elements. There is only one way to do that keeps the reference counts valid)


c) For user-defined managed, not reference counted types, it would just do

  for i := 0 to count - 1 do dest[i] := source[i]

Because you really want to use Move for strings, it can be vastly faster 
than updating the refcounts. But it might not work with user-defined 
types, like someone could build a managed type that cannot be moved. E.g.:


 type TA = record
  class operator Initialize(var a: TA);
  procedure dosomething;
end;

var globalPointer: ^TA;
class operator TA.Initialize(var a: TA);
begin
  globalPointer := @a;
end;
procedure TA.dosomething;
begin
  assert(globalPointer = @self)
end;




And when you use IsManagedType, it does not distinguish standard strings 
with such weird managed types.


And perhaps there could be a special attribute to mark which kind of 
moving is needed, e.g..


  type [moveable] TA = record
  type [referencecounted] TA = record
  type [nonmoveable] TA = record

Bye,
Benito
On 09.01.21 16:28, Bart via fpc-pascal wrote:

Hi,

This may be a silly question.
I use System.Move() to move items in a dynamic array, like
   Move(FData[0], FData[OldEnd], FStart*SizeOf(T));
Where T is the type of the elements in the array.
This seems to work as expected.

I have some questions though:

1. Does this depend on the alignment of the array?
2. Is it OK if the elements of the array are (or contain) managed types?
3. Are there caveats if T is a specialization of a generic type definition?

___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

[fpc-pascal] CP_NONE string disappearing

2020-12-27 Thread Benito van der Zander via fpc-pascal


Hello,

this code:

program Project1;
{$mode objfpc}{$H+}
var a,b, c: string;
begin
  a := 'x';
  b := 'y';
  SetCodePage(RawByteString(b), CP_NONE, false);
  c := a+b;
  writeln(c);
end.

prints x without y on win32/wine.

Is that supposed to happen?



Bye,

Benito


___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

Re: [fpc-pascal] I'm working on automated Help Output for console apps

2020-11-20 Thread Benito van der Zander via fpc-pascal


Hi,

I also made such a thing:

var optionsreader: TCommandLineReader;
begin
  optionsreader := TCommandLineReader.create;

  optionsreader.declareFile('file', 'The file to be processed');
  optionsreader.addAbbreviation('f');

  optionsreader.declareFlag('help', '');
  optionsreader.addAbbreviation('h');

  optionsreader.declareFlag('version', 'Print the version of the 
application');

  optionsreader.addAbbreviation('v');

  if optionsreader.existsProperty('help') then
     writeln(optionsreader.availableOptions);

end.


Output:

--file=  or -f    The file to be processed
--help or -h
--version or -v     Print the version of the application


You can download it at http://www.benibela.de/sources_en.html#rcmdline

Benito

On 20.11.20 01:33, Graeme Geldenhuys via fpc-pascal wrote:

Hi,

I'm working on a automated help output writer(s) for console apps.
Thus no more tedious and ugly output when you do: myapp -h

My aims:
  * I write a lot of console apps, so this would be very useful to me.
  * Ability to swap out the help formatter. It's interface based, so
custom implementations can easily be created.
  * Auto align help options and descriptions - the most annoying thing to
do manually. ;-)
  * Ability to define console width in which to format and wrap the help
output, but has a default value.
  * The idea is loosely base on the Java version found in Apache Commons.

When it's done I'll obviously shared it as open source somewhere.

With that said, below is how I currently use it. It uses the Builder design
pattern so gives it the Chain Invocations syntax. I know it's not something
often seen in Pascal programs, but it makes it very easy to read and easy
to use/type, especially with code completion editors like Lazarus IDE.

For those console programmers out there... Is there anything in console help
output that you like or wish you had. That way I could possibly add it and
make this even more useful to a wider audience.

I'm still working on AppName, Version and Usage output.

Example code:
==
var
   optionlist: TOptions;
   helpFormatter: IHelpFormatter;
   header: string;
   footer: string;
begin
   optionlist := TOptions.Create;

   optionlist.add(TOption.Builder
 .isRequired
 .withDescription('The file to be processed')
 .hasArg
 .withArgName('file')
 .withLongOpt('file')
 .build('f'));// build() always takes the mandatory short 
option.

   optionlist.add(TOption.Builder
 .withLongOpt('help')
 .withArgName('test')  // this is ignored because .hasArg was not 
specified
 .build('h'));

   optionlist.add(TOption.Builder
 .withDescription('Print the version of the application')
 .withLongOpt('version')
 .build('v'));

   header := 'Do something useful with an input file' + LineEnding;
   footer := LineEnding + 'Please report issues at http://example.com/issues';

   helpFormatter := TBasicHelpFormatter.Create();

   // sample outputs with increasing verbosity

   writeln('===   (1)');
   helpFormatter.printHelp(optionlist);

   writeln('===   (2)');
   helpFormatter.printHelp('DocView v1.0', optionlist);

   writeln('===   (3)');
   helpFormatter.printHelp('DocView v1.0', header, optionlist, footer);

   writeln('== the end =');
   optionlist.Free;
end;
==


And here is the example output for the 3 options so far:

===   (1)
* -f,--file The file to be processed
   -h,--help
   -v,--versionPrint the version of the application

* indicates required parameters
===   (2)
DocView v1.0
* -f,--file The file to be processed
   -h,--help
   -v,--versionPrint the version of the application

* indicates required parameters
===   (3)
DocView v1.0
Do something useful with an input file

* -f,--file The file to be processed
   -h,--help
   -v,--versionPrint the version of the application

* indicates required parameters

Please report issues at http://example.com/issues
== the end =



Regards,
   Graeme

___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

Re: [fpc-pascal] link.res search dirs when cross compiling [solved]

2020-11-18 Thread Benito van der Zander via fpc-pascal



Some are hardcoded, and fpc.cfg might contain -Fl lines with paths too 
from parsing ld.so.conf on install

Well, I did not put them in ~/.fpc.cfg
In /etc/fpc.cfg there is

-Fl/usr/lib/gcc/x86_64-linux-gnu/10
-Fl/usr/lib/x86_64-linux-gnu/fpc/$fpcversion/lib/$FPCTARGET
-Fl/usr/lib/$fpctarget-*

The last one explains it. I actually have a directory 
/usr/lib/'${DEB_HOST_MULTIARCH}'.  Why would Ubuntu even ship such a 
directory.


$ dpkg -S DEB_HOST_MULTIARCH
libnss3:i386, libnss3:amd64: 
/usr/lib/${DEB_HOST_MULTIARCH}/libfreeblpriv3.chk
libnss3:i386, libnss3:amd64: 
/usr/lib/${DEB_HOST_MULTIARCH}/libfreeblpriv3.so

libnss3:i386, libnss3:amd64: /usr/lib/${DEB_HOST_MULTIARCH}/libfreebl3.chk
libnss3:i386, libnss3:amd64: /usr/lib/${DEB_HOST_MULTIARCH}/libfreebl3.so
libnss3:i386, libnss3:amd64: /usr/lib/${DEB_HOST_MULTIARCH}


Benito



On 18.11.20 20:33, Marco van de Voort via fpc-pascal wrote:


Op 2020-11-18 om 19:21 schreef Benito van der Zander via fpc-pascal:


where do the SEARCH_DIRs in link.res come from?


Some are hardcoded, and fpc.cfg might contain -Fl lines with paths too 
from parsing ld.so.conf on install



___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

Re: [fpc-pascal] link.res search dirs when cross compiling

2020-11-18 Thread Benito van der Zander via fpc-pascal


How are you specifying your cross-compilation sysroot? Make sure you use
the -XR parameter for that,


I use -Fl in ~/.fpc.cfg...


When I use -XR nothing happens. The link.res is the same, except -Fl 
prepends the path to link.res and -XR just ignores it. (same for -Xr)



Benito


___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

[fpc-pascal] link.res search dirs when cross compiling

2020-11-18 Thread Benito van der Zander via fpc-pascal


Hi,


where do the SEARCH_DIRs in link.res come from?

My build has started failing, after trying it on a new computer with 
"/home/benito/bin/arm-linux-androideabi-ld.bfd:android/libs/armeabi/link42815.res:17: 
ignoring invalid character `'' in expression"


link.res starts with:

SEARCH_DIR(/home/benito/opt/android/sdk/ndk-bundle/platforms/android-21/arch-arm/usr/lib/)
SEARCH_DIR(/usr/lib/thunderbird-addons/)
SEARCH_DIR(/usr/lib/modules/)
SEARCH_DIR(/usr/lib/aspell/)
SEARCH_DIR(/usr/lib/gnome-shell/)
SEARCH_DIR(/usr/lib/binfmt-support/)
SEARCH_DIR(/usr/lib/udev/)
SEARCH_DIR(/usr/lib/gcc/)
SEARCH_DIR(/usr/lib/cups/)
SEARCH_DIR(/usr/lib/tmpfiles.d/)
SEARCH_DIR(/usr/lib/firewalld/)
SEARCH_DIR(/usr/lib/X11/)
SEARCH_DIR(/usr/lib/cmake/)
SEARCH_DIR(/usr/lib/realmd/)
SEARCH_DIR(/usr/lib/klibc/)
SEARCH_DIR(/usr/lib/sasl2/)
SEARCH_DIR('/usr/lib/${DEB_HOST_MULTIARCH}/')

...


I guess '/usr/lib/${DEB_HOST_MULTIARCH}/ is the problem.  Why would it 
even add such a dir



Benito

___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

Re: [fpc-pascal] Adding file to string to the RTL

2020-10-09 Thread Benito van der Zander via fpc-pascal







They cannot be used on handles that do not support FileSeek() 
(sockets, pipes, stdin/stdout etc.).



Well, it would be better if it could

You can just incrementally resize the return array, when reading 
succeeds after seeking fails.


I have a string load function doing that:

https://github.com/benibela/bbutils/blob/master/bbutils.pas#L3652





  Is it possible to extend this same set of functions to writing? 
Naturally I need to write back to the file now and the same problem

presents itself.  I have to search through other code bases to find a
function or Google and find 



Writing a file should write the data in a temporary file and then rename 
the temporary file to replace the target file. Otherwise it might 
destroy the target file, without writing the new content, when there is 
an error. Although not even renaming is always safe, some people say you 
need a filesystem-specific transaction log.




Benito

On 06.10.2020 10:12, Michael Van Canneyt via fpc-pascal wrote:




No, we don't deal in magic, only bits and bytes :-)

I added the following functions to the sysutils unit (rev 47056):

// Read raw content as bytes

Function GetFileContents(Const aFileName : RawByteString) : TBytes;
Function GetFileContents(Const aFileName : UnicodeString) : TBytes;
Function GetFileContents(Const aHandle : THandle) : TBytes;

// Read content as string

// Assume TEncoding.SystemEncoding
Function GetFileAsString(Const aFileName : RawByteString) : 
RawByteString;

// Specify encoding
Function GetFileAsString(Const aFileName : RawByteString; aEncoding : 
TEncoding) : RawByteString;

// Assume TEncoding.Unicode contents
Function GetFileAsString(Const aFileName : UnicodeString) : 
UnicodeString;

// Specify encoding, return Unicode string.
Function GetFileAsString(Const aFileName : UnicodeString; aEncoding : 
TEncoding) : UnicodeString;


These functions will raise an exception if the file cannot be opened 
or read.
They cannot be used on handles that do not support FileSeek() 
(sockets, pipes, stdin/stdout etc.).


I did some tests on encoding conversion but not extensively. If you 
find any errors, please report them through the bugtracker.


Michael.
___
fpc-pascal maillist  - fpc-pascal@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal


___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

[fpc-pascal] json parsing: detecting invalid escape sequences

2020-09-29 Thread Benito van der Zander via fpc-pascal


Hi,

I am supposed to find invalid escape sequences when parsing JSON and 
replace them with a user defined fallback. Invalid in the sense that the 
unicode codepoint is not defined or a missing surrogate, not 
syntactically invalid.


For example, any occurrence of \u and \uDEAD should be replaced by 
\u and \udead respectively. Or alternatively with  depending on 
the settings.


I think I need to change the JSON scanner to be able to do that.

I could add a callback function OnInvalidEscape: function (escapeStart: 
pchar): string; of object;
Or perhaps OnInvalidEscape: function (unicodePoint, 
previousUnicodePointSurrogate: integer): string; of object; {although 
that would be troublesome if \uDEAD and \udead are supposed to be 
replaced with a different fallback}
Or OnInvalidEscape: function (const escapedString: string[4]): string; 
of object;


The function would return the unescaped value. Alternatively, the 
current string could be passed to it as var parameter, and the function 
would append its unescaped value directly.


Or move all unescaping to a callback function, could be called 
OnUnescape or OnDecodeEscape. So the scanner does not need to decide 
which escapes are invalid. Then


  if (joUTF8 in Options) or 
(DefaultSystemCodePage=CP_UTF8) then
S:=Utf8Encode(WideString(WideChar(u1)+WideChar(u2))) // ToDo: use faster 
function

  else
    S:=String(WideChar(u1)+WideChar(u2)); // 
WideChar converts the encoding. Should it warn on loss?


could be replaced by one function call. And if the user does not set a 
callback function, the scanner would set its own callback function 
depending on the option.


Any interest in a patch that adds such a callback function? Or is there 
another way to do this?


Best,
Benito
___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

[fpc-pascal] json numbers with leading dots

2020-09-29 Thread Benito van der Zander via fpc-pascal


Hi,
there are also two lines in the json scanner where it tries to repair 
numbers with leading dots '.123' to '0.123':


 If (FCurTokenString[1]='.') then
  FCurTokenString:='0'+FCurTokenString;

They should probably be removed. Not only are those numbers invalid in 
json, it is also very slow to allocate a new string. And StrToFloat 
works with '.123', so it should not change anything.


Although removing them would break programs that cannot handle '.123', 
they are broken anyways, because it is not adding a zero to  '-.123'.



Best,
Benito
___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

Re: [fpc-pascal] json parser line numbers

2020-09-29 Thread Benito van der Zander via fpc-pascal



here: https://bugs.freepascal.org/view.php?id=37836


On 29.09.20 10:47, Michael Van Canneyt via fpc-pascal wrote:



On Tue, 29 Sep 2020, Benito van der Zander via fpc-pascal wrote:


Hi,

the line numbering of the json parser has been changed recently.

It used to say "Error at line 1"... when there was an error in the 
first line, but now it says "Error at line 0"...


Was that on purpose, or can someone change it back?


It was not on purpose. Please file a bugreport.

Michael.
___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

[fpc-pascal] json parser line numbers

2020-09-29 Thread Benito van der Zander via fpc-pascal


Hi,

the line numbering of the json parser has been changed recently.

It used to say "Error at line 1"... when there was an error in the first 
line, but now it says "Error at line 0"...


Was that on purpose, or can someone change it back?

Benito

___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

Re: [fpc-pascal] String literals and code page of .pas source file

2020-09-14 Thread Benito van der Zander via fpc-pascal



Hi,



I would definitely keep it that way.

As I see it: Redirection or not should not matter, the system should 
assume console output.

Things like 'tee' make this concept dubious in any case:

If you pipe output to a program, you don't expect the codepage to change
because of the redirection. 



In my projects, I set the output codepage to utf-8 when there is a 
redirection


It is platform agnostic, on identical inputs, it creates identical 
output files on Linux and Windows. No one needs OEM files on Linux


Best,
Benito

On 14.09.2020 14:51, Michael Van Canneyt via fpc-pascal wrote:



On Mon, 14 Sep 2020, Tomas Hajny via fpc-pascal wrote:


opened:
{$ifdef FPC_HAS_FEATURE_ANSISTRINGS}
  { if no codepage is yet assigned then assign default ansi codepage }
TextRec(t).CodePage:=TranslatePlaceholderCP(TextRec(t).CodePage);
{$else FPC_HAS_FEATURE_ANSISTRINGS}

I see no need to change this ?


Please, have a look at OpenStdIO (implemented at the end of 
text.inc). That one changes the default codepage for Input, Output 
and StdErr, but it doesn't care about possible redirection and/or 
piping to some other application (not necessarily written in FPC). 
That approach has both advantages and disadvantages. The advantage is 
that "type " will give correct results on the 
console. The disadvantage is that opening the redirected file with 
any Win32 GUI application (let's say notepad.exe) will result in 
garbage.  I don't say that it is necessarily bad, but it should be 
documented at least if we want to keep it that way.


I would definitely keep it that way.

As I see it: Redirection or not should not matter, the system should 
assume console output.

Things like 'tee' make this concept dubious in any case:

If you pipe output to a program, you don't expect the codepage to change
because of the redirection.


I think it will differ since Crt is not codepage aware. If you want 
it to
work the same you'll have to make Crt codepage (and hence unicode) 
aware.


As mentioned by me, Crt is currently more codepage aware than the 
System unit output as far as output to console is concerned, because 
Crt provides correct output even for shortstrings (unlike the System 
unit).


I would need to check the details, but that sounds more like 
accidental behaviour as opposed to intended behaviour to me :-)


Michael.
___
fpc-pascal maillist  - fpc-pascal@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal


___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

Re: [fpc-pascal] Procedural generics question

2020-08-25 Thread Benito van der Zander via fpc-pascal


Hi,

that is generating rather odd code (r40721)

project1.lpr:9    begin
00401090 55   push   %rbp
00401091 4889e5   mov    %rsp,%rbp
00401094 488d6424f0   lea -0x10(%rsp),%rsp
00401099 48895df8 mov    %rbx,-0x8(%rbp)
project1.lpr:10   if TypeInfo(T) = 
TypeInfo(Integer) then WriteLn('an integer');
0040109D 488d1584af0700   lea 0x7af84(%rip),%rdx    
# 0x47c028 
004010A4 488d057daf0700   lea 0x7af7d(%rip),%rax    
# 0x47c028 

004010AB 4839c2   cmp    %rax,%rdx
004010AE 752b jne    0x4010db 


004010B0 e8cbcf0100   callq  0x41e080 
004010B5 4889c3   mov    %rax,%rbx
004010B8 488d15412f0600   lea 0x62f41(%rip),%rdx    
# 0x464000 <_$PROJECT1$_Ld1>

004010BF 4889de   mov    %rbx,%rsi
004010C2 31ff xor    %edi,%edi
004010C4 e867d30100   callq  0x41e430 


004010C9 e8d2700100   callq  0x4181a0 
004010CE 4889df   mov    %rbx,%rdi
004010D1 e85ad20100   callq  0x41e330 
004010D6 e8c5700100   callq  0x4181a0 
project1.lpr:11   if TypeInfo(T) = 
TypeInfo(String) then WriteLn('a string');
004010DB 488d0546af0700   lea 0x7af46(%rip),%rax    
# 0x47c028 
004010E2 488d1537b20700   lea 0x7b237(%rip),%rdx    
# 0x47c320 

004010E9 4839d0   cmp    %rdx,%rax
004010EC 752b jne    0x401119 


004010EE e88dcf0100   callq  0x41e080 
004010F3 4889c3   mov    %rax,%rbx
004010F6 488d15132f0600   lea 0x62f13(%rip),%rdx    
# 0x464010 <_$PROJECT1$_Ld2>

004010FD 4889de   mov    %rbx,%rsi
00401100 31ff xor    %edi,%edi
00401102 e829d30100   callq  0x41e430 


00401107 e894700100   callq  0x4181a0 
0040110C 4889df   mov    %rbx,%rdi
0040110F e81cd20100   callq  0x41e330 
00401114 e887700100   callq  0x4181a0 
project1.lpr:12   end;

The compiler should know everything about TypeInfo(T) and optimize it 
away where possible


Cheers,
Benito
On 25.08.20 04:32, leledumbo via fpc-pascal wrote:

I remember something like this in RTTI though but can't find it in your

docs yet for generics.

{$mode objfpc}

uses
   typinfo;

generic procedure Add;
begin
if TypeInfo(T) = TypeInfo(Integer) then WriteLn('an integer');
if TypeInfo(T) = TypeInfo(String) then WriteLn('a string');
end;

begin
   specialize Add;
   specialize Add;
end.



--
Sent from: http://free-pascal-general.1045716.n5.nabble.com/
___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

38 matches

Mail list logo