Re: [fpc-devel] Some questions about compiler work on x86_64-win64
First of all sorry that it take so long time for my answer. 03/19/17 10:38:35, Sven Barth via fpc-devel: > If it is so important to you: patches are welcome. But keep in mind that the default needs to be the status quo. Thanks. This is important to me to know that changes that i want to see, can be accepted at all (even if i need wrote patch myself). I already start work on this patch and have first results, but i need ask some aditional questions about how better do it. > Since ASLR is disabled by default in FPC that question is useless. This why a i offer change default behaviour. For EXE-files on windows generate not rip-related code for access global varibles and change default image base to $40. For DLL`s and EXE-files with "-WR" (without it ASLR not work in any case) switch generate rip-related code and automaticaly set ASLR (or even high-entropy 64-bit address space layout randomization) flags. This make situation clear. Or static image base and take all benefits from it, or ASLR and take benefits from it (now we take only bad thinks from both). As my fast check show FPC RTL can work in ASLR mode, so looks like no problems with it. About "dumanic packages"... Its take some time for me, too consider it and i have some idea. But for it, i start other topic. Plus i have little request, if you or someone else have a time, please check this patch http://bugs.freepascal.org/view.php?id=29894 and say, is it can be accepted or not. And is this future request assepteble or not http://bugs.freepascal.org/view.php?id=31124 . If it accepteble, even if FPC team will not work on this, i will try make it (possible not so fast, but i try do it with other optimizations that i need for my project). Regards, Roman ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
Re: [fpc-devel] Some questions about compiler work on x86_64-win64
Am 18.03.2017 23:11 schrieb "Bishop": > > 03/18/17 00:51:05, Sven Barth via fpc-devel < fpc-devel@lists.freepascal.org>: > > > > Bo, the main sense of this is to detect when a new thread is started and more importantly terminated cause only with this we can free the threadvar area of the thread accordingly (if the thread is an external one, not one started using BeginThread or TThread). > > Thanks, now i understand how its work. Plus as i understand on Linux (and other unixes) threadvar for external threads allocated on first access to them (and free via PThread ability to call destructor for key). Correct. (and my first word should have been "No", not "Bo"; stupid smartphone keyboard) > > > Why *should* it be auto generated if one can use a table and let the RTL do the rest. > Is it not better make all that can be done in compile time? Its not more complex solution for compiler code, but as i see it, its more harmonious (Its depend not only INIFINAL, but all tables, than used in RTL to make work of compiler/linker. As example, FPC_THREADVARTABLES. Different modules, i mean DLL or SO, use different TLS keys for their threadvar regions. But why position of variable from begin of threadvar region must be generated in runtime? Isn`t it work for linker?). Possible this is depend on that "dynamic packages"? If you have different modules (the binary and the libraries) then they are *separate* entities. Cause it could be that a Pascal library is used with a C binary and thus a library has the whole RTL statically linked (or at least that part that is used). Only dynamic packages allow one to transparently have units be part of different binary modules yet providing one whole application. Package libraries can however only be used by a binary compiled with the same compiler as they rely on quite a bit of compiler magic. > > > Also with the addition of dynamic packages this will move even more towards a table based approach. > Where i can read information about what is it and why we need it? What kind of problems is must solve? Because we already have dll/so, and as i know and see for now its enough. Possible my knowledge is not enough to see whole problem. With dynamic packages you can share classes, strings, memory, etc. between the modules (the main binary and the different package libraries), because the RTL will only exist once. And all this transparently for the user. When you use ordinary libraries you need to use a shared memory manager to pass strings around and you can't use the "as" and "is" operators inside the main binary on classes passed in from the libraries (and by extension this also applies to exceptions). > > > But you can set the corresponding PE flag for ASLR using $SetPEOpts (or so). No recompilation needed in that case. > Can. But what if i dont want ASLR binary? Its totaly valid. Since ASLR is disabled by default in FPC that question is useless. > > > Microsoft recommended that approach for Win64 so why should we do the work and implement it differently even if ASLR isn't enabled by default for FPC executables? > Recommendation in not a law (like it is with SEH in Win64). C compilers allow both type of programs, depend on what programmer need. Is it need many work to change it? As i see it, its just one check in compiler code for global varibles (if select PIC - use RIP-related, if not - use direct). It already done in linux. I think it was better to give compiler user more possibilities when its cost almoust nothing. If it is so important to you: patches are welcome. But keep in mind that the default needs to be the status quo. Regards, Sven ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
Re: [fpc-devel] Some questions about compiler work on x86_64-win64
03/18/17 00:51:05, Sven Barth via fpc-devel: > Bo, the main sense of this is to detect when a new thread is started and more importantly terminated cause only with this we can free the threadvar area of the thread accordingly (if the thread is an external one, not one started using BeginThread or TThread). Thanks, now i understand how its work. Plus as i understand on Linux (and other unixes) threadvar for external threads allocated on first access to them (and free via PThread ability to call destructor for key). > Why *should* it be auto generated if one can use a table and let the RTL do the rest. Is it not better make all that can be done in compile time? Its not more complex solution for compiler code, but as i see it, its more harmonious (Its depend not only INIFINAL, but all tables, than used in RTL to make work of compiler/linker. As example, FPC_THREADVARTABLES. Different modules, i mean DLL or SO, use different TLS keys for their threadvar regions. But why position of variable from begin of threadvar region must be generated in runtime? Isn`t it work for linker?). Possible this is depend on that "dynamic packages"? > Also with the addition of dynamic packages this will move even more towards a table based approach. Where i can read information about what is it and why we need it? What kind of problems is must solve? Because we already have dll/so, and as i know and see for now its enough. Possible my knowledge is not enough to see whole problem. > But you can set the corresponding PE flag for ASLR using $SetPEOpts (or so). No recompilation needed in that case. Can. But what if i dont want ASLR binary? Its totaly valid. > Microsoft recommended that approach for Win64 so why should we do the work and implement it differently even if ASLR isn't enabled by default for FPC executables? Recommendation in not a law (like it is with SEH in Win64). C compilers allow both type of programs, depend on what programmer need. Is it need many work to change it? As i see it, its just one check in compiler code for global varibles (if select PIC - use RIP-related, if not - use direct). It already done in linux. I think it was better to give compiler user more possibilities when its cost almoust nothing. Regards, Roman ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
Re: [fpc-devel] Some questions about compiler work on x86_64-win64
Am 17.03.2017 22:16 schrieb "Bishop": > > 03/17/17 11:56:47, Sven Barth via fpc-devel < fpc-devel@lists.freepascal.org>: > > >> 1) EXE-file contained .CRT section. As i understand it contained one pointer to _FPC_Tls_Callback functions from RTL. Is it used only for this? > > Correct. > And main sence of this function is allocated ThreadVar regions for thread, than was created in initialization code of DLL`s that was loaded staticaly (OS load them use information in .idata section)? Bo, the main sense of this is to detect when a new thread is started and more importantly terminated cause only with this we can free the threadvar area of the thread accordingly (if the thread is an external one, not one started using BeginThread or TThread). > > > It probably should be .rdata, but I think that older binutils had problems with that, thus they were put into .data and that was never changed till now... > So now it is possible to change it? Plus what about 2nd part of my question? > > > >Or even possible it must be evicted and replace just with call instructions? I mean autogeneration of fpc_InitializeUnits and fpc_FinalizeUnits? > What reason for we need so strange init/final unit function calling solution? Autogenerated functions will take less space (direct or RIP-related address - all functions in same segment), safer than varible in w/r segment (this is a way to make some atack on application) and faster. Why *should* it be auto generated if one can use a table and let the RTL do the rest. Also with the addition of dynamic packages this will move even more towards a table based approach. > > > I might remember it incorrectly, but I think that Microsoft says that all accesses should be RIP relative to reduce the need for relocations. > But EXE generated by FPC dont ASLR aware. This have no needed flags in EXE file header (I use VMMap to get info about process and it show ASLR modules). Windows support both type of executable (i dont say about DLL in this time). Static and ASLR. I think it will be logicaly if FPC can generate both type of applications (it already have flags for PIC and RIP-related addressing is a good form of PIC realization). Plus on Linux we have picture like it. Main binary can be PIC or not-PIC. SO-libraries can be only PIC (that logicaly because they need relocation, that exe-file can avoid). But you can set the corresponding PE flag for ASLR using $SetPEOpts (or so). No recompilation needed in that case. Microsoft recommended that approach for Win64 so why should we do the work and implement it differently even if ASLR isn't enabled by default for FPC executables? Regards, Sven ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
Re: [fpc-devel] Some questions about compiler work on x86_64-win64
03/17/17 11:56:47, Sven Barth via fpc-devel: >> 1) EXE-file contained .CRT section. As i understand it contained one pointer to _FPC_Tls_Callback functions from RTL. Is it used only for this? > Correct. And main sence of this function is allocated ThreadVar regions for thread, than was created in initialization code of DLL`s that was loaded staticaly (OS load them use information in .idata section)? > It probably should be .rdata, but I think that older binutils had problems with that, thus they were put into .data and that was never changed till now... So now it is possible to change it? Plus what about 2nd part of my question? >Or even possible it must be evicted and replace just with call instructions? I mean autogeneration of fpc_InitializeUnits and fpc_FinalizeUnits? What reason for we need so strange init/final unit function calling solution? Autogenerated functions will take less space (direct or RIP-related address - all functions in same segment), safer than varible in w/r segment (this is a way to make some atack on application) and faster. > I might remember it incorrectly, but I think that Microsoft says that all accesses should be RIP relative to reduce the need for relocations. But EXE generated by FPC dont ASLR aware. This have no needed flags in EXE file header (I use VMMap to get info about process and it show ASLR modules). Windows support both type of executable (i dont say about DLL in this time). Static and ASLR. I think it will be logicaly if FPC can generate both type of applications (it already have flags for PIC and RIP-related addressing is a good form of PIC realization). Plus on Linux we have picture like it. Main binary can be PIC or not-PIC. SO-libraries can be only PIC (that logicaly because they need relocation, that exe-file can avoid). Regards, Roman ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
Re: [fpc-devel] Some questions about compiler work on x86_64-win64
Am 17.03.2017 08:08 schrieb "Bishop via fpc-devel" < fpc-devel@lists.freepascal.org>: > > I have some questions about compiler work on x86_64-win64. > > 1) EXE-file contained .CRT section. As i understand it contained one pointer to _FPC_Tls_Callback functions from RTL. Is it used only for this? Correct. > 2) Why INITFINAL table on both Windows and Linux puted to .data section? Is we need to change it in time of app run? Or even possible it must be evicted and replace just with call instructions? I mean autogeneration of fpc_InitializeUnits and fpc_FinalizeUnits? It probably should be .rdata, but I think that older binutils had problems with that, thus they were put into .data and that was never changed till now... > 3) Why .fpc.n_version puted to .data section? Is we need change it on runtime? Possible must be in .rdata? On linux it is entyre section for it, does this is necessary? Same as above. > 4) Why referencing to global variable under Windows all time compiled like RIP-related, even if no Generate-PIC enable (on Linux work fine)? I might remember it incorrectly, but I think that Microsoft says that all accesses should be RIP relative to reduce the need for relocations. Regards, Sven ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel