Re: [fpc-pascal] Candidate for crowd-funding
> On Dec 20, 2018, at 1:17 AM, Karoly Balogh (Charlie/SGR) > wrote: > > No, that assembly function just does swapping of the stack and storing the > caller's stack settings and setting up the coroutine's stack instead. The > actual stack is allocated with the Win32 API function VirtualAlloc(), the > rest is just structure copy to/from the coroutine function and setting > up/restoring the stack. The stack is just a memory area, where the > stackpointer points (plus on some archs it has to respect certain > alignment requirements, but that's easy enough to deal with), and until > your stackpointer register points to that memory area, the rest just > works, due to how the CPU works, the compiled code just uses it... The > compiler doesn't handle it normally on the "allocation" level, as usually > the OS takes care of it. It sounds like a sound theory but I don’t have the expertise to do the assembly involved for saving/restoring the stack. I think the RTL would need some extra functions to pair with SetJmp/LongJmp but if someone could provide those I’d be happy to work on the other parts involved. Regards, Ryan Joseph ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] Candidate for crowd-funding
Hi, On Wed, 19 Dec 2018, Ryan Joseph wrote: > > (After reading that old thread we had...) > > > > Actually, it can. The only thing you need to do is to allocate a stack > > first, then make the "coroutine" functions use it, which is basically > > boils down to how you pass your arguments and self instance to the > > coroutine on initializatin, while you change the stack pointer register, > > and then how you restore it in the end. > > And how is a stack allocated? I assume we’re talking assembly here? The > link I posted (http://www.festra.com/wwwboard/messages/12899.html) seems > to do this but I don’t know assembly so I’m just guessing. No, that assembly function just does swapping of the stack and storing the caller's stack settings and setting up the coroutine's stack instead. The actual stack is allocated with the Win32 API function VirtualAlloc(), the rest is just structure copy to/from the coroutine function and setting up/restoring the stack. The stack is just a memory area, where the stackpointer points (plus on some archs it has to respect certain alignment requirements, but that's easy enough to deal with), and until your stackpointer register points to that memory area, the rest just works, due to how the CPU works, the compiled code just uses it... The compiler doesn't handle it normally on the "allocation" level, as usually the OS takes care of it. > Even if you allocated a stack frame doesn’t it need to be populated with > all local variables in the current scope? In my short time working on > the compiler I haven’t looked at the actual code generator so I don’t > know how the stack works. However, in theory you should be able to make > a compiler intrinsic that wraps this process for you. Well, this is quite a complex code, so most likely would end up as a helper in the RTL anyway... Charlie___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] Candidate for crowd-funding
> On Dec 19, 2018, at 9:28 PM, Karoly Balogh (Charlie/SGR) > wrote: > > (After reading that old thread we had...) > > Actually, it can. The only thing you need to do is to allocate a stack > first, then make the "coroutine" functions use it, which is basically > boils down to how you pass your arguments and self instance to the > coroutine on initializatin, while you change the stack pointer register, > and then how you restore it in the end. And how is a stack allocated? I assume we’re talking assembly here? The link I posted (http://www.festra.com/wwwboard/messages/12899.html) seems to do this but I don’t know assembly so I’m just guessing. Even if you allocated a stack frame doesn’t it need to be populated with all local variables in the current scope? In my short time working on the compiler I haven’t looked at the actual code generator so I don’t know how the stack works. However, in theory you should be able to make a compiler intrinsic that wraps this process for you. Regards, Ryan Joseph ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] Candidate for crowd-funding
Hi, On Wed, 19 Dec 2018, Ryan Joseph wrote: > > On Dec 19, 2018, at 5:23 PM, Sven Barth via fpc-pascal > > wrote: > > > > A library as first step would be more useful. Later on one can think > > about integrating it into the language itself. > > > > How does that work? I’ve been interested in coroutines for a while now > but I don’t understand how they function in the language. They would > need to manage stack states and push/restore stacks, which is something > a library can’t do right? (After reading that old thread we had...) Actually, it can. The only thing you need to do is to allocate a stack first, then make the "coroutine" functions use it, which is basically boils down to how you pass your arguments and self instance to the coroutine on initializatin, while you change the stack pointer register, and then how you restore it in the end. It might be messy, with small bits in platform-specific assembly - actually the compiler or the RTL could indeed help a bit there, but this is not a requirement - but theoretically possible, on the other hand I have absolutely no clue from the top of my head how that would interfere with any sort of debug info generation, and/or exception handling. Actually, if you check the Amiga startup code in the RTL, that already does this, as AmigaOS and similar systems have a single address space approach, with a limited size stack, so the RTL startup code checks if the stack is big enough, and allocates a new one if it is too limited for the application to run. The OS provides the necessary stack-swap functionality there, while storing the old stack values for reuse in a small structure, but this code could just as well be in the application, or your library in this case. Then the code utilizes setjmp/longjmp to restore register state on exit, before it quits and restores the old stack. https://github.com/graemeg/freepascal/blob/master/rtl/morphos/si_prc.pp#L47 https://github.com/graemeg/freepascal/blob/master/rtl/amiga/m68k/si_prc.pp#L60 So to sum it up, after you set up a separate stack and stack pointer register to your coroutine, the previously discussed setjmp/longjmp method should work for switching between them. (Also, if you set up some example-library framework, I might be able to help with getting the stackswap thing running, if you are not too comfortable with that. Even if I think this entire coroutine idea is just a too big of a can of worms to integrate into a compiler/language and a standard library, but as a separate library it might have its uses indeed.) Cheers, -- Charlie___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] Candidate for crowd-funding
Am Mi., 19. Dez. 2018, 11:38 hat Martok geschrieben: > > The main problem here is to model what registers an instruction uses and > > modifies so that the register allocator of the surrounding function can > take > > that into account accordingly... Maybe as a first step we could allow in > lining > > for funcrions that have a register clause with the touched registers... > > But yes, in the long term this would definitely be nice. > > I actually did that a few months ago (when intrinsics came up on this > list), and > it does work as you expect. The actual difficult problem is argument > location > remapping. Other than that, inline-nostackframe-asm[registers] needs > almost no > changes. > Maybe we could add your changes to trunk? Regards, Sven > ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] Candidate for crowd-funding
Am Mi., 19. Dez. 2018, 13:18 hat Ryan Joseph geschrieben: > > > > On Dec 19, 2018, at 6:35 PM, Ryan Joseph > wrote: > > > > How does that work? I’ve been interested in coroutines for a while now > but I don’t understand how they function in the language. They would need > to manage stack states and push/restore stacks, which is something a > library can’t do right? > > I already discussed this on length a while ago it seems. :) > > > http://free-pascal-general.1045716.n5.nabble.com/Coroutines-and-VirtualAlloc-td5728274.html > > There was no conclusion on how this would be implemented, let alone in a > library form, but there was an old link which showed an implementation > using assembly mixed in. I don’t know assembly though so it doesn’t inform > me too much. > > http://www.festra.com/wwwboard/messages/12899.html I never said that it would be easy, but it's definitely possible as there also exist C/C++ libraries for coroutines. Regards, Sven ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] Candidate for crowd-funding
> On Dec 19, 2018, at 6:35 PM, Ryan Joseph wrote: > > How does that work? I’ve been interested in coroutines for a while now but I > don’t understand how they function in the language. They would need to manage > stack states and push/restore stacks, which is something a library can’t do > right? I already discussed this on length a while ago it seems. :) http://free-pascal-general.1045716.n5.nabble.com/Coroutines-and-VirtualAlloc-td5728274.html There was no conclusion on how this would be implemented, let alone in a library form, but there was an old link which showed an implementation using assembly mixed in. I don’t know assembly though so it doesn’t inform me too much. http://www.festra.com/wwwboard/messages/12899.html Regards, Ryan Joseph ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] Candidate for crowd-funding
> On Dec 19, 2018, at 5:23 PM, Sven Barth via fpc-pascal > wrote: > > A library as first step would be more useful. Later on one can think about > integrating it into the language itself. > How does that work? I’ve been interested in coroutines for a while now but I don’t understand how they function in the language. They would need to manage stack states and push/restore stacks, which is something a library can’t do right? Regards, Ryan Joseph ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] Candidate for crowd-funding
El 19/12/18 a las 09:02, denisgolovan escribió: Hi all I decided to start a separate thread for asking about potential candidate for crowd-funding. My personal wish-list is: - support for array calculations / automatic loop parallelization via SSE, AVX, etc. Both static and dynamic arrays should supported. Once implemented vector operations on arrays (ala APL) might be done using operator overloads. - Custom/separate allocators for dynamic arrays (to avoid manually patching compiler). - Coroutines. Portable library or in-compiler support. - Interprocedural optimizations (something akin to LTO) - inline assembler function support - proper macro language perhaps Could someone comment if those goal are attractive to somebody else? I mean both donators and potential "implementors". BTW, is it possible to state the specific project when donating? Mine: - Debugger: Watch values of properties of classes/objects even when they have a getter and/or are string Those above, for me, would improve Lazarus 1%. More than any improvement in package management, IDE or changes in language. For me, they are must. These below would be a good improvement - Debugger: Setting values of properties of classes/objects even when they have a setter and/or are string (not as important as watching, but very important as well) - Debugger: Watch values of variables in nested procedures. - Debugger: better dealing with generics Amazing enough, there is no project in then foundation related to improve debugger. Am I the only one who thinks that poor debugger is a stopper? -- Saludos Santiago A. ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] Candidate for crowd-funding
>> - Custom/separate allocators for dynamic arrays (to avoid manually patching >> compiler). > > Why do you need that? Besides ordinary arrays I allocate/use arrays backed by mmaped files. Existing functions taking arrays as arguments mostly remain working transparently. That's a huge win in code size and allow zero-copy scenarios. Besides that, I am able provide special alignment for arrays. That also would be necessary to vector/SSE/AVX work. >> - inline assembler function support > > The main problem here is to model what registers an instruction uses and > modifies so that the register allocator of the surrounding function can take > that into account accordingly... Maybe as a first step we could allow in > lining for funcrions that have a register clause with the touched registers... > But yes, in the long term this would definitely be nice. Agree. Register clause will ease a lot of pain. >> - proper macro language perhaps > > No. We already rejected such an idea some months ago. Ok. Looks like m4 will be my friend forever :) -- Regards, Denis Golovan ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] Candidate for crowd-funding
> The main problem here is to model what registers an instruction uses and > modifies so that the register allocator of the surrounding function can take > that into account accordingly... Maybe as a first step we could allow in > lining > for funcrions that have a register clause with the touched registers... > But yes, in the long term this would definitely be nice. I actually did that a few months ago (when intrinsics came up on this list), and it does work as you expect. The actual difficult problem is argument location remapping. Other than that, inline-nostackframe-asm[registers] needs almost no changes. -- Regards, Martok ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] Candidate for crowd-funding
Am Mi., 19. Dez. 2018, 09:03 hat denisgolovan geschrieben: > My personal wish-list is: > - support for array calculations / automatic loop parallelization via SSE, > AVX, etc. > Both static and dynamic arrays should supported. > Once implemented vector operations on arrays (ala APL) might be done > using operator overloads. > There is already some vector operation optimizations (though right now I don't know the specifics when they're triggered) and there were some plans to improve on that. So, yes, this one is welcome. - Custom/separate allocators for dynamic arrays (to avoid manually patching > compiler). > Why do you need that? - Coroutines. Portable library or in-compiler support. > A library as first step would be more useful. Later on one can think about integrating it into the language itself. - Interprocedural optimizations (something akin to LTO) > We already have WPO which does some things, so that can be build upon. Though it's always a two step approach. Implementing interprocedural optimizations on the node level might be interesting, though for now inline helps there... - inline assembler function support > The main problem here is to model what registers an instruction uses and modifies so that the register allocator of the surrounding function can take that into account accordingly... Maybe as a first step we could allow in lining for funcrions that have a register clause with the touched registers... But yes, in the long term this would definitely be nice. - proper macro language perhaps > No. We already rejected such an idea some months ago. Regards, Sven ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] Candidate for crowd-funding
> I will leave the technical comments to the compiler developers. > >> BTW, is it possible to state the specific project when donating? > > Yes, if I recall correctly you can give a message when the paypal donate > page appears. (just as you can state that you want to be in the hall of > fame) Ok. Thanks. -- Regards, Denis Golovan ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] Candidate for crowd-funding
> Hi all > > I decided to start a separate thread for asking about potential candidate > for crowd-funding. > > My personal wish-list is: > - support for array calculations / automatic loop parallelization via SSE, > AVX, etc. > Both static and dynamic arrays should supported. > Once implemented vector operations on arrays (ala APL) might be done > using operator overloads. > - Custom/separate allocators for dynamic arrays (to avoid manually > patching compiler). > - Coroutines. Portable library or in-compiler support. > - Interprocedural optimizations (something akin to LTO) > - inline assembler function support > - proper macro language perhaps > > Could someone comment if those goal are attractive to somebody else? > I mean both donators and potential "implementors". I will leave the technical comments to the compiler developers. > > BTW, is it possible to state the specific project when donating? Yes, if I recall correctly you can give a message when the paypal donate page appears. (just as you can state that you want to be in the hall of fame) Maybe I should state this more explicit on the website. But you can also simply contact me or the chairman. Michael. Michael. ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
[fpc-pascal] Candidate for crowd-funding
Hi all I decided to start a separate thread for asking about potential candidate for crowd-funding. My personal wish-list is: - support for array calculations / automatic loop parallelization via SSE, AVX, etc. Both static and dynamic arrays should supported. Once implemented vector operations on arrays (ala APL) might be done using operator overloads. - Custom/separate allocators for dynamic arrays (to avoid manually patching compiler). - Coroutines. Portable library or in-compiler support. - Interprocedural optimizations (something akin to LTO) - inline assembler function support - proper macro language perhaps Could someone comment if those goal are attractive to somebody else? I mean both donators and potential "implementors". BTW, is it possible to state the specific project when donating? -- Regards, Denis Golovan ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
