Unsubscribe
Good bye Rand Dow :-( You are now unsubscribed -- UNSUBSCRIBE: mailto:picolisp@software-lab.de?subject=Unsubscribe
Re: Are PicoLisp's coroutine stacks legal?
Hi Alex, On Tue, May 21, 2013 at 7:46 AM, Alexander Burger a...@software-lab.dewrote: To make things more clear, let's consider the following situation. It is after the program started two coroutines, and then execution continued in the main program. The two coroutines are suspended, and the hardware stack points into the main program's stack frame: +---+ Stack frame #1| | Main program | | |Stack | |Area | - SP (Stack pointer register points here) | | ... | e.g. 4 MB| | | | | | | +---+ Stack frame #2| tag | Coroutine #1 +---+ | Saved SP| +--+ This is the SP of the suspended coroutine +---+| | || | Environment || | || +---+| | || | || |Stack || |Area | --+ | | ... | e.g. 1 MB| | | | | | | +---+ Stack frame #3| tag | Coroutine #2 +---+ | Saved SP| +--+ This is the SP of the suspended coroutine +---+| | || | Environment || | || +---+| | || | || | | --+ ... Is that picture a single, contiguous memory region? Because then, you *could* have the first scenario I mentioned in a previous mail (unlikely, but, think of a native C code, that allocated huge structures and recursed -- it could overwrite many megabytes of stack). This would be my concern: a C library function that uses (in the above picture) more than 4 MB of memory. That is easy to imagine with modern reentrant graphics libraries. The programmer would have to have a very clear picture of the above to know to use the 'stack' function to allocate adequate stack. Anyway, enough is *not ever* enough. I would only feel comfortable seeing at least 10MB of *non-mapped* memory between the main program's stack and each of the the co-routines' stacks. Again, generally, I would feel uncomfortable having anything below my stack pointer than a large region of writable memory followed by a very large region of memory that would give me a sigsegv if I tried to access it. If I am the application programmer, I can violate these rules, but as the language system programmer (i.e., you) you can never make assumptions about how that application programmer is going to use the stack. Now, if the system inspects this process (during a system call or interrupt), it finds that - despite the fact that the process ONCE used a large stack space - the stack pointer is now back in the upper regions of the stack space. The system doesn't know about the frame structures. It will assume that the process did a lot of pushes and/or recursive calls, but is now done with them, and the stack is no longer needed. So it might decide to purge it. The big question is: Will the system (which system?) do that? If so, how do other languages implement green threads, coroutines or continuations? Since that is read/write memory, the system won't purge it. It might be paged out (to swap), but would be transparently paged back in if/when used again. I'd never expect an OS to know that a region of memory is a stack. Rand
Re: Are PicoLisp's coroutine stacks legal?
Hi Alex, Yes, by all means, keep the coroutines! Perhaps add some of this mailing list chain as a caveat about the stack in the coroutines documentation: http://picolisp.com/5000/!wiki?coroutines In particular, the picture of the stack and the warning to allocate enough stack if there is any chance of using a lot of stack. Rand On Tue, May 21, 2013 at 6:55 PM, Alexander Burger a...@software-lab.dewrote: Yep. That's the trade-off. As soon as you use a coroutine in PicoLisp, you have to be conscious about your stack. For a program that has no coroutine active when a lot of stack space is needed, these limitations don't apply. As soon as the last coroutine finishes or is stopped, the stack becomes unlimited again (at least if you called ulimit -s unlimited). ... Then you could call (stack 3) before starting the first coroutine. This allocates 12 MB for the main task, and 3 MB for each coroutine.
Re: Are PicoLisp's coroutine stacks legal?
I won't try to point to documentation Each co-routine should have it's own separate stack. Best practices with stack management today have a sufficiently large stack that grows and then terminates in unmapped memory. If it is attempted to grow the stack too much, then the program gets a memory exception (SIGSEGV on systems that use it). Think about a recursive routine with relatively large local data structures (stack-based). Such a routine could overwrite any amount of stack, finish, and then return with no errors. The current PicoLisp implementation sounds like, then, when a co-routine switch occurs, the switched to routine could find it's stack corrupted. And certainly operating system events could overwrite things deeper on the stack during a context switch. Rand On Mon, May 20, 2013 at 6:43 PM, Alexander Burger a...@software-lab.dewrote: Hi all, since nearly three years PicoLisp supports coroutines (64-bit version). Now suddenly it occurred to me that the way I implemented them might be illegal. The problem is how individual stacks for the coroutines are allocated. I do this by reserving space on the stack (by decrementing the stack pointer) upon starting a new coroutine. Execution of that coroutine may be suspended (with 'yield', so that execution continues in the main program or some other coroutine), to be resumed later. Switching between individual (co)routines is done by moving the stack pointer up and down between the corresponding stack frames. Everything seems to work fine - though I haven't used coroutines in serious projects yet. Now I remember to have read that (in POSIX?) no valid data should be stored *below* the current position of the stack pointer. Is that really the case? I can't find any conclusive information about that. This switching between stack frames, as described above, of course leaves temporarily suspended stack frames *below* the stack pointer, while it points into a higher frame. Does this mean that the operating system might dispose or overwrite them? For example, when paging is required? Interrupts are probably not a problem, as they push their contexts on the system stack. Some sources say that even GCC stores (small amounts of) data below the stack pointer, while others (possibly some ABIs from Intel) say this is an absolute no-no. Can anybody point to a definite answer? ♪♫ Alex -- UNSUBSCRIBE: mailto:picolisp@software-lab.de?subject=Unsubscribe
Re: In Detail: Native C Calls
Alex, That looks great! I'm very impressed with native and now he doc! Cheers! Rand On May 9, 2012, at 3:33 PM, Alexander Burger a...@software-lab.de wrote: Hi all, at last, I have found the time to write an in-detail description of the 'native' function: http://software-lab.de/doc/native.html Any comments welcome! It became quite long, not because 'native' is so complicated, but because there are so many ways of passing information to and from C functions. I hope it clears some of the smoke. Does anybody know of a programming language with an equally powerful -- yet as simple -- C function interface? Except C/C++ of course ;-) Cheers, - Alex -- UNSUBSCRIBE: mailto:picolisp@software-lab.de?subject=Unsubscribe -- UNSUBSCRIBE: mailto:picolisp@software-lab.de?subject=Unsubscribe
