Re: [fpc-devel] FPC/Delphi/FastMM4/TopMemory speed test
Adem wrote: I did some rough + emprical (i.e. non-exhaustive + non-authoritative) tests with the native memory managers of -- x64 FPC (v2.4.3)/Lazarus (v0.9.29-26670), -- Delphi (2010), So this is 32 bit vs 64 bit? Micha ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
[fpc-devel] Need advice for refactoring
Since some time I'm trying to separate the syntax from the semantics processing in the parser. It turned out to be quite complicated, so that I want to use some methodology. (Yes, I've been warned ;-) For profiling and debugging I want to have both the old and new code in the same executable file, so that a commandline option or environment variable can be used to select which "version" to use. Is this feasable, WRT the invocation of fpc with e.g. the test suite? How would you do it? Furthermore I have no clue how to use new compiler at all. Do I have to rebuild fpc, or how else is the fpc executable (or link?) related to concrete (machine/target specific) compilers? The separation shall be done in multiple steps. In the first step I'll enclose all semantical code into begin-end, so that it can be easily hidden (folded) in the Lazarus IDE. Later these blocks shall be extended into if cond then new_proc else begin [old code, also copied into new_proc()] end; This has the advantage over $IF... that the IDE allows to navigate to everything in both if-else branches. It also should allow to apply patches (from updated trunk), to some degree, because the code sequence is not much modified. The new_proc procedures will be local subroutines first, which can share the local variables of the enclosing parse_... procedure. Later these subroutines will be moved into different units, where they finally are exchangable by changing the searchpath. Here again it's helpful to view both versions in different editor windows, side by side. For first results see the dodi/parser_rewrite branch. Perhaps I missed more issues? Any hints are welcome :-) DoDi ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
[fpc-devel] FPC/Delphi/FastMM4/TopMemory speed test
I did some rough + emprical (i.e. non-exhaustive + non-authoritative) tests with the native memory managers of -- x64 FPC (v2.4.3)/Lazarus (v0.9.29-26670), -- Delphi (2010), -- FastMM4 v4.94 -- TopMemory v3.54. Test application is fairly simple: Using a double-linked list, it generated 1,000,000 nodes and freed them. I repeated these 12 times. I ignored duplicate speed values [also ignored negative resuls, see note 2 below.] Timing values were obtained by the use of this function: function GetCPUTicks: Int64; asm RDTSC; end FPC timings: 481.88; 483.29; 483.78; 484.37; 484.60; 485.01; 485.88; 487.01; 488.43; 491.07; 492.20; 525.44 FPC average: 489.42 TopMemory timings: 411.83; 412.52; 412.63; 412.83; 413.30; 413.72; 414.00; 414.25; 414.62; 415.94; 416.05; 553.15 TopMemory average: 425.40 Delphi timings: 301.06; 302.19; 302.35; 302.49; 302.62; 302.68; 302.94; 303.74; 304.18; 306.45; 308.02; 309.28 TopMemory average: 304 FastMM4 timings: 279.74; 282.08; 282.26; 282.37; 282.56; 282.76; 282.80; 282.81; 282.97; 283.09; 283.60; 284.18 FastMM4 average: 282.60 Here are the ratios I ended up with. Delphi/FastMM4 = 1.0757 FPC/Delphi = 1.6099 FPC/FastMM4 = 1.7318 FPC/TopMemory = 1.1504 TopMemory/Delphi = 1.3993 TopMemory/FastMM4 = 1.5053 From these, it's clear that FastMM (latest version) is the fastest. As far as FPC is concerned, switching to TopMemory would mean a speed loss of %15 --for my test application. Notes: 1) These were the compiler options used: -MObjFPC -Scghi -O3 -OoREGVAR -gl -WG -vewnhi -l -Fu. -dLCL -dLCLwin32 2) Under FPC I got some negative values between a GetCPUTicks and another one taken later. Even though I ignored those values, I'd love to know why/how that happens. ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Purpose of "uses ... in"?
On Wed, Jul 14, 2010 at 6:35 PM, Stefan Kisdaroczi wrote:
> I'm still using it. Or make a symlink:
> ln -s rar/zip.pas rar.pas
>
> In combination with {$L xzy.o} and 'external name ...' you can do
> wonderful things. Don't do this at home! (maybe at work :-)
Okay... this is very sophisticated. I have not used it (yet) but this
is not so simple.
I still thinking my sugestion (in other mail, and copied below) is a good way:
uses
zip IN '/units/my/zip.pas' AS myzip;
zip IN '/units/lib/zip/zip.pas' AS ziplib;
2 units, same name, different paths and different "aliases" to use in project.
On Wed, Jul 14, 2010 at 7:06 PM, José Mejuto wrote:
> I can vaguely remember a case like this in Delphi several years ago,
> but I think the problem was a precompiled .dcu which I can not rename.
> Is this possible ?
...but my sugestion do no serves for compiled files... or serves?
Regards,
MD.
___
fpc-devel maillist - fpc-devel@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Access violation building the compiler : r15585
Le 16 juillet 2010 15:33:50 UTC, Inoussa OUEDRAOGO a écrit : > > > 2010/7/16 Jonas Maebe : >> Inoussa OUEDRAOGO wrote on Fri, 16 Jul 2010: >> >>> I just check out and can no longer build the compiler. >> >> While it's probably due to a commit of mine, I can't debug it because >> everything works fine on Mac OS X and Linux. > > How can I help ? Note that I have no knowledge of the compiler source code. -- Inoussa O. ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Access violation building the compiler : r15585
2010/7/16 Jonas Maebe : > Inoussa OUEDRAOGO wrote on Fri, 16 Jul 2010: > >> I just check out and can no longer build the compiler. > > While it's probably due to a commit of mine, I can't debug it because > everything works fine on Mac OS X and Linux. How can I help ? -- Inoussa O. ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Access violation building the compiler : r15585
Inoussa OUEDRAOGO wrote on Fri, 16 Jul 2010: I just check out and can no longer build the compiler. While it's probably due to a commit of mine, I can't debug it because everything works fine on Mac OS X and Linux. Jonas This message was sent using IMP, the Internet Messaging Program. ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
[fpc-devel] Unit loading overhead
Florian Klaempfl wrote on Fri, 16 Jul 2010: One of the bottlenecks the common user encounters, is unit loading: especially projects like the lazarus suffer from the time spent into unit loading while I suspect that it narrows down also to procedures like fillchar which consume a lot of time. The main slowdown when recompiling projects is that FPC often recompiles or re-resolves the same unit multiple times when a unit in its uses clause has changed. The ppu loading itself is quite fast. Recompiling Lazarus without changing any unit just takes 2.2 seconds on my machine (without assembling/linking). Compiling program using all units from the packages dir (910 units) takes 4.4 seconds (without assembling/linking). The following result is from compiling a program that uses 348 (precompiled) units from the packages tree on darwin/x86-64 and lists all functions taking up 1% or more of the total execution time (sample-based). I didn't use all units here because then my laptop does not keep all ppu files in the disk cache during the profiling and that obviously skews the results. 7.6%mach_kernel vm_map_enter 4.0%ppcx48 FPC_MOVE 3.9%ppcx48 CCLASSES_FPHASH$SHORTSTRING$$LONGWORD 3.6%mach_kernel blkclr 3.1%mach_kernel vm_map_lookup_entry 2.4%ppcx48 SYSTEM_SYSGETMEM_FIXED$QWORD$$POINTER 1.9%ppcx48 SYSTEM_SYSFREEMEM_FIXED$PFREELISTS$PMEMCHUNK_FIXED$$QWORD 1.8%mach_kernel ml_set_interrupts_enabled 1.7%ppcx48 SYSTEM_ALLOC_OSCHUNK$PFREELISTS$QWORD$QWORD$$POINTER 1.7%mach_kernel lo_alltraps 1.6%ppcx48 FPC_ANSISTR_DECR_REF 1.5%libSystem.B.dylib __bzero 1.4%ppcx48 SYSTEM_SYSFREEMEM$POINTER$$QWORD 1.4%ppcx48 fpc_pushexceptaddr 1.2%ppcx48 SYSTEM_REMOVE_FREED_FIXED_CHUNKS$POSCHUNK 1.1%ppcx48 CCLASSES_TDYNAMICARRAY_$__READ$formal$LONGWORD$$LONGWORD 1.1%ppcx48 SYMTYPE_TDEREF_$__RESOLVE$$TOBJECT 1.1%mach_kernel pmap_enter 1.1%ppcx48 fpc_popaddrstack 1.1%ppcx48 SYSTEM_TOBJECT_$__NEWINSTANCE$$TOBJECT 1.1%mach_kernel pmap_remove_range 1.0%mach_kernel cache_lookup_path vmmap_enter is from mmap. This can be improved by increasing the blocksize used to initialise pools for small blocks from 32Kb to 256Kb (to support this for 32 bit systems, fixedoffsetshift in rtl/inc/heap.inc has to be changed from 16 to 12, which is no problem since only the 4 lowest bits are currently used for flags). 5.1% ppcx49 FPC_MOVE // source: 1.3% fpc_shortstr_to_shortstr, 1.1% ppufile.readdata, 0.5% fpc_ansistr_copy 3.7% mach_kernel blkclr // kernel zeroing pages when we mmap memory and it has no reserve zeroed pages 3.6%ppcx49 SYSTEM_SYSGETMEM_FIXED$QWORD$$POINTER 3.5%ppcx49 CCLASSES_FPHASH$SHORTSTRING$$LONGWORD 2.2%ppcx49 SYSTEM_SYSFREEMEM_FIXED$PFREELISTS$PMEMCHUNK_FIXED$$QWORD 2.1%libSystem.B.dylib __bzero // fillchar(0) 2.0%ppcx49 SYSTEM_REMOVE_FREED_FIXED_CHUNKS$POSCHUNK 1.9%ppcx49 SYSTEM_ALLOC_OSCHUNK$PFREELISTS$QWORD$QWORD$$POINTER 1.8%ppcx49 FPC_ANSISTR_DECR_REF 1.8%ppcx49 SYSTEM_SYSFREEMEM$POINTER$$QWORD 1.7%mach_kernel lo_alltraps 1.6%mach_kernel ml_set_interrupts_enabled 1.4%ppcx49 SYMTYPE_TDEREF_$__RESOLVE$$TOBJECT 1.4%mach_kernel pmap_enter // page fault 1.4%ppcx49 fpc_pushexceptaddr 1.4%ppcx49 SYSUTILS_COMPARETEXT$ANSISTRING$ANSISTRING$$LONGINT 1.2%mach_kernel pmap_remove_range // munmap 1.1%ppcx49 PPU_TPPUFILE_$__GETBYTE$$BYTE 1.1%ppcx49 CCLASSES_TFPHASHLIST_$__INTERNALFIND$LONGWORD$SHORTSTRING$LONGINT$$LONGINT 1.1%mach_kernel vm_page_lookup // page fault 1.1%ppcx49 SYSTEM_SETJMP$JMP_BUF$$LONGINT 1.0%ppcx49 FPC_MOVE 1.0%mach_kernel vm_map_enter // mmap 1.0%ppcx49 SYSTEM_TOBJECT_$__NEWINSTANCE$$TOBJECT 1.0%ppcx49 SYSTEM_SYSGETMEM_VAR$QWORD$$POINTER 1.0%ppcx49 fpc_varset_add_sets 1.0%ppcx49 FPC_SHORTSTR_COMPARE_EQUAL 1.0%ppcx49 fpc_ansistr_setlength In real time (without assembling/linking): Before After user0m1.621s user 0m1.636s sys 0m0.791s sys0m0.492s Total memory usage barely changes (from 297MB to 299MB). I guess it's no problem to commit this, but in most cases it probably won't change much if anything performance-wise unless you do almost nothing but allocate tons of small memory blocks without every freeing any in between. Jonas This message was sent using IMP, the Internet Messaging Program. ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Messages overhead
Op Fri, 16 Jul 2010, schreef Micha Nelissen: Daniël Mantione wrote: Op Fri, 16 Jul 2010, schreef Micha Nelissen: Does FPC compile significantly faster when using -O- then? That is irrelevant; for example the register allocator made the compiler slower (initially even 3 times, but we managed to reduce it to a few ten %), but had great effects on code quality. What do you mean with code quality then? I thought performance of the generated code. Then we are thinking the same :) The register allocator sacrified compiler speed for code quality. 2.0 generates much better code than 1.0. Also there were lots of bugs due to runing out of registers before, the register allocator dealt with that problem once and forever, so you can also say we sacrified compiler speed for compiler reliability. Daniël___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Messages overhead
Daniël Mantione wrote: Op Fri, 16 Jul 2010, schreef Micha Nelissen: Does FPC compile significantly faster when using -O- then? That is irrelevant; for example the register allocator made the compiler slower (initially even 3 times, but we managed to reduce it to a few ten %), but had great effects on code quality. What do you mean with code quality then? I thought performance of the generated code. Micha ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Messages overhead
Op Fri, 16 Jul 2010, schreef Micha Nelissen: Daniël Mantione wrote: Compiler speed can loose from maintainability, but it can also loose from code quality; the performance of your application is probably also worth a lot to you. Does FPC compile significantly faster when using -O- then? That is irrelevant; for example the register allocator made the compiler slower (initially even 3 times, but we managed to reduce it to a few ten %), but had great effects on code quality. The most expensive optimization you can effect with -O is the assembler optimizer, this is noticable in compilation speed. However, debug info is for example something that affects speed more. Daniël___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Messages overhead
Daniël Mantione wrote: Compiler speed can loose from maintainability, but it can also loose from code quality; the performance of your application is probably also worth a lot to you. Does FPC compile significantly faster when using -O- then? Micha ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
[fpc-devel] Access violation building the compiler : r15585
Hi I just check out and can no longer build the compiler. This is the used command : PATH=E:\FPC\2.4.0\bin\i386-win32;%PATH% make all PP=E:\FPC\2.4.0\bin\i386-win32\ppc386.exe >make_all_prefix_res.txt and the (the last lines of) output make -C win32 all make[7]: Entering directory `E:/fpc_dev/rtl/win32' E:/fpc_dev/compiler/ppc2.exe -Ur -Xs -O2 -n -Fi../inc -Fi../i386 -Fi../win -FE. -FUE:/fpc_dev/rtl/units/i386-win32 -di386 -dRELEASE -Us -Sg system.pp -Fi../win Fatal: Compilation aborted An unhandled exception occurred at $0046A164 : EAccessViolation : Access violation $0046A164 $00498027 $0049569D $004E3087 $004E3379 $004E36BB $004E00B0 $004E2B1F $0042F71E $0041137C $00401806 -- Inoussa O. ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Re: Compiler bottlenecks
On 07/15/2010 04:43 PM, Hans-Peter Diettrich wrote: Just an idea: When the lists contain many entries, they could be split into buckets. Then the currently searched bucket(s) could be locked against use by other threads, which can skip them and inspect the next bucket. I suppose their idea is that concurrent requests are so seldom that it is not helpful to implement anything better than just using medium size chunks. _Detecting_ the conflict is another issue, as potentially necessary atomic instruction might slow down the program a lot. -Michael ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Re: Compiler bottlenecks
On 07/15/2010 05:17 PM, Adem wrote: I was curious about the differences about FastMM (which I use under Delphi) and TopMM, and asked about it in Delphi ThirdPartyTools NG. Thus TopMM seems excellently suited here. I don't understand why in their Graphics, they imply that the Delphi 7 project uses FastMM. Does TopMM have a license that allows for using it in The FP RTL ? -Michael ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
[fpc-devel] How to get function start and end position in executable?
Hello all, I want to implement kind of executable protection using "software guards" (*) - when function (caller) decrypts another function (callee) before calling it and encrypts it again after callee returns. As a result I need to get addresses of my functions (where the asm code starts and ends in the executable). Is there some way to export such info to external file while fpc compiles executable (or maybe afterwards to get this info from executable)? Any suggenstions would be appreciated! (*) J. Cappaert, N. Kisserli, D. Schellekens, B. Preneel. "Self-encrypting Code to Protect Against Analysis and Tampering", 2006 (https://www.cosic.esat.kuleuven.be/publications/article-811.pdf) Regards Žilvinas Ledas ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Messages overhead
Op 2010-07-15 23:50, Daniël Mantione het geskryf: > No. That is doesn't do justice to all the effort that is put into > performance optimization. It's not about maintainability being more > important. > > It is about making the right trade-offs between: > - Compiler speed > - Compiler memory usage > - Generated code quality > - Compiler portability > - And indeed compiler maintainability. This is kind of what I meant, I just put it in much simpler terms. Sorry if my earlier statement caused offence. > Nevertheless contest the idea that FPC is a slow compiler, I have put a I never said FPC is slow, just that it is "slower" than Kylix and Delphi - but I also tried to explained why it is slower. I do know and acknoledge that FPC is MUCH faster than many other language compilers out there. A testament to the great work you guys hove done. Regards, - Graeme - -- fpGUI Toolkit - a cross-platform GUI toolkit using Free Pascal http://opensoft.homeip.net/fpgui/ ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
