Re: core.thread.Fiber --- runtime stack overflow unlike goroutines

2014-09-21 Thread Martin Nowak via Digitalmars-d-learn
Am I missing something? Is there a clean and simple way to get Fiber to no longer suffer a stack overflow when implementing D-routines? Simply choose a big enough stack size when creating your fibers. http://dlang.org/library/core/thread/Fiber.this.html It's fairly cheap to use a really big

Re: core.thread.Fiber --- runtime stack overflow unlike goroutines

2014-09-21 Thread Martin Nowak via Digitalmars-d-learn
On 08/15/2014 05:09 PM, Sean Kelly wrote: At least on OSX, it appears that mapping memory is constant time regardless of size, but there is some max total memory I'm allowed to map, presumably based on the size of a vmm lookup tabe. The max block size I can allocate is 1 GB, and I can allocate

Re: core.thread.Fiber --- runtime stack overflow unlike goroutines

2014-08-22 Thread Dicebot via Digitalmars-d-learn
On Friday, 15 August 2014 at 15:50:30 UTC, Dicebot wrote: On Friday, 15 August 2014 at 15:40:35 UTC, Sean Kelly wrote: On Friday, 15 August 2014 at 15:25:23 UTC, Dicebot wrote: No, I was referring to the proposal to supply bigger stack size to Fiber constructor - AFAIR it currently does

Re: core.thread.Fiber --- runtime stack overflow unlike goroutines

2014-08-16 Thread Kagamin via Digitalmars-d-learn
On Friday, 15 August 2014 at 22:26:54 UTC, ketmar via Digitalmars-d-learn wrote: and we -- 32-bit addicts -- will run out of address space while 64-bit happy people will laugh looking at us. ;-) You should only choose stack size carefully or keep data in TempAlloc instead of stack.

Re: core.thread.Fiber --- runtime stack overflow unlike goroutines

2014-08-16 Thread ketmar via Digitalmars-d-learn
On Sat, 16 Aug 2014 17:49:28 + Kagamin via Digitalmars-d-learn digitalmars-d-learn@puremagic.com wrote: You should only choose stack size carefully or keep data in TempAlloc instead of stack. but i can choose stack size carefully without such 'address reserving' feature. ;-)

Re: core.thread.Fiber --- runtime stack overflow unlike goroutines

2014-08-15 Thread Kagamin via Digitalmars-d-learn
On Thursday, 14 August 2014 at 18:52:00 UTC, Sean Kelly wrote: On 64 bit, reserve a huge chunk of memory, set a SEGV handler and commit more as needed. Basically how kernel thread stacks work. I've been meaning to do this but haven't gotten around to it yet. AFAIK, OS already provides this

Re: core.thread.Fiber --- runtime stack overflow unlike goroutines

2014-08-15 Thread Kagamin via Digitalmars-d-learn
http://msdn.microsoft.com/en-us/library/windows/desktop/aa366887%28v=vs.85%29.aspx Allocates memory charges (from the overall size of memory and the paging files on disk) for the specified reserved memory pages. The function also guarantees that when the caller later initially accesses the

Re: core.thread.Fiber --- runtime stack overflow unlike goroutines

2014-08-15 Thread Kagamin via Digitalmars-d-learn
On Thursday, 14 August 2014 at 07:46:29 UTC, Carl Sturtivant wrote: The default size of the runtime stack for a Fiber is 4*PAGESIZE which is very small, and a quick test shows that a Fiber suffers a stack overflow that doesn't lead to a clean termination when this limit is exceeded. Pass a

Re: core.thread.Fiber --- runtime stack overflow unlike goroutines

2014-08-15 Thread Sean Kelly via Digitalmars-d-learn
On Friday, 15 August 2014 at 08:36:34 UTC, Kagamin wrote: http://msdn.microsoft.com/en-us/library/windows/desktop/aa366887%28v=vs.85%29.aspx Allocates memory charges (from the overall size of memory and the paging files on disk) for the specified reserved memory pages. The function also

Re: core.thread.Fiber --- runtime stack overflow unlike goroutines

2014-08-15 Thread Dicebot via Digitalmars-d-learn
On Friday, 15 August 2014 at 08:41:30 UTC, Kagamin wrote: On Thursday, 14 August 2014 at 07:46:29 UTC, Carl Sturtivant wrote: The default size of the runtime stack for a Fiber is 4*PAGESIZE which is very small, and a quick test shows that a Fiber suffers a stack overflow that doesn't lead to a

Re: core.thread.Fiber --- runtime stack overflow unlike goroutines

2014-08-15 Thread Kagamin via Digitalmars-d-learn
On Friday, 15 August 2014 at 14:26:28 UTC, Sean Kelly wrote: Oh handy, so there's basically no work to be done on Windows. I'll have to check the behavior of mmap on Posix. I heard, calloc behaves this way on linux (COW blank page mapped to the entire range), it was discussed here some time

Re: core.thread.Fiber --- runtime stack overflow unlike goroutines

2014-08-15 Thread Kagamin via Digitalmars-d-learn
On Friday, 15 August 2014 at 14:28:34 UTC, Kagamin wrote: On Friday, 15 August 2014 at 14:26:28 UTC, Sean Kelly wrote: Oh handy, so there's basically no work to be done on Windows. I'll have to check the behavior of mmap on Posix. I heard, calloc behaves this way on linux (COW blank page

Re: core.thread.Fiber --- runtime stack overflow unlike goroutines

2014-08-15 Thread Sean Kelly via Digitalmars-d-learn
On Friday, 15 August 2014 at 14:28:34 UTC, Dicebot wrote: Won't that kind of kill the purpose of Fiber as low-cost context abstraction? Stack size does add up for thousands of fibers. As long as allocation speed is fast for large allocs (which I have to test), I want to change the default

Re: core.thread.Fiber --- runtime stack overflow unlike goroutines

2014-08-15 Thread Kagamin via Digitalmars-d-learn
On Friday, 15 August 2014 at 14:28:34 UTC, Dicebot wrote: Won't that kind of kill the purpose of Fiber as low-cost context abstraction? Stack size does add up for thousands of fibers. I didn't measure it.

Re: core.thread.Fiber --- runtime stack overflow unlike goroutines

2014-08-15 Thread Sean Kelly via Digitalmars-d-learn
On Friday, 15 August 2014 at 14:26:28 UTC, Sean Kelly wrote: On Friday, 15 August 2014 at 08:36:34 UTC, Kagamin wrote: http://msdn.microsoft.com/en-us/library/windows/desktop/aa366887%28v=vs.85%29.aspx Allocates memory charges (from the overall size of memory and the paging files on disk) for

Re: core.thread.Fiber --- runtime stack overflow unlike goroutines

2014-08-15 Thread Sean Kelly via Digitalmars-d-learn
At least on OSX, it appears that mapping memory is constant time regardless of size, but there is some max total memory I'm allowed to map, presumably based on the size of a vmm lookup tabe. The max block size I can allocate is 1 GB, and I can allocate roughly 131,000 of these blocks before

Re: core.thread.Fiber --- runtime stack overflow unlike goroutines

2014-08-15 Thread Dicebot via Digitalmars-d-learn
On Friday, 15 August 2014 at 14:45:02 UTC, Sean Kelly wrote: On Friday, 15 August 2014 at 14:28:34 UTC, Dicebot wrote: Won't that kind of kill the purpose of Fiber as low-cost context abstraction? Stack size does add up for thousands of fibers. As long as allocation speed is fast for large

Re: core.thread.Fiber --- runtime stack overflow unlike goroutines

2014-08-15 Thread Sean Kelly via Digitalmars-d-learn
On Friday, 15 August 2014 at 15:25:23 UTC, Dicebot wrote: No, I was referring to the proposal to supply bigger stack size to Fiber constructor - AFAIR it currently does allocate that memory eagerly (and does not use any OS CoW tools), doesn't it? I thought it did, but apparently the

Re: core.thread.Fiber --- runtime stack overflow unlike goroutines

2014-08-15 Thread Dicebot via Digitalmars-d-learn
On Friday, 15 August 2014 at 15:40:35 UTC, Sean Kelly wrote: On Friday, 15 August 2014 at 15:25:23 UTC, Dicebot wrote: No, I was referring to the proposal to supply bigger stack size to Fiber constructor - AFAIR it currently does allocate that memory eagerly (and does not use any OS CoW

Re: core.thread.Fiber --- runtime stack overflow unlike goroutines

2014-08-15 Thread Carl Sturtivant via Digitalmars-d-learn
On Thursday, 14 August 2014 at 18:52:00 UTC, Sean Kelly wrote: On 64 bit, reserve a huge chunk of memory, set a SEGV handler and commit more as needed. Basically how kernel thread stacks work. I've been meaning to do this but haven't gotten around to it yet. Very nice; the hardware VM

Re: core.thread.Fiber --- runtime stack overflow unlike goroutines

2014-08-15 Thread Carl Sturtivant via Digitalmars-d-learn
On Friday, 15 August 2014 at 08:41:30 UTC, Kagamin wrote: On Thursday, 14 August 2014 at 07:46:29 UTC, Carl Sturtivant wrote: The default size of the runtime stack for a Fiber is 4*PAGESIZE which is very small, and a quick test shows that a Fiber suffers a stack overflow that doesn't lead to a

Re: core.thread.Fiber --- runtime stack overflow unlike goroutines

2014-08-15 Thread Carl Sturtivant via Digitalmars-d-learn
On Friday, 15 August 2014 at 20:11:43 UTC, Carl Sturtivant wrote: On Friday, 15 August 2014 at 08:41:30 UTC, Kagamin wrote: On Thursday, 14 August 2014 at 07:46:29 UTC, Carl Sturtivant wrote: The default size of the runtime stack for a Fiber is 4*PAGESIZE which is very small, and a quick test

Re: core.thread.Fiber --- runtime stack overflow unlike goroutines

2014-08-15 Thread Carl Sturtivant via Digitalmars-d-learn
On Friday, 15 August 2014 at 15:40:35 UTC, Sean Kelly wrote: On Friday, 15 August 2014 at 15:25:23 UTC, Dicebot wrote: No, I was referring to the proposal to supply bigger stack size to Fiber constructor - AFAIR it currently does allocate that memory eagerly (and does not use any OS CoW

Re: core.thread.Fiber --- runtime stack overflow unlike goroutines

2014-08-15 Thread Sean Kelly via Digitalmars-d-learn
On Friday, 15 August 2014 at 20:17:51 UTC, Carl Sturtivant wrote: On Friday, 15 August 2014 at 15:40:35 UTC, Sean Kelly wrote: I thought it did, but apparently the behavior of VirtualAlloc and mmap (which Fiber uses to allocate the stack) simply reserves the range and then commits it lazily,

Re: core.thread.Fiber --- runtime stack overflow unlike goroutines

2014-08-15 Thread ketmar via Digitalmars-d-learn
On Fri, 15 Aug 2014 20:19:18 + Carl Sturtivant via Digitalmars-d-learn digitalmars-d-learn@puremagic.com wrote: Should have read further down the thread --- you're right as the memory is in effect merely reserved virtual memory and isn't actually allocated. and we -- 32-bit addicts --

core.thread.Fiber --- runtime stack overflow unlike goroutines

2014-08-14 Thread Carl Sturtivant via Digitalmars-d-learn
The default size of the runtime stack for a Fiber is 4*PAGESIZE which is very small, and a quick test shows that a Fiber suffers a stack overflow that doesn't lead to a clean termination when this limit is exceeded. This makes it difficult to simulate deterministic alternation where the

Re: core.thread.Fiber --- runtime stack overflow unlike goroutines

2014-08-14 Thread Sean Kelly via Digitalmars-d-learn
On 64 bit, reserve a huge chunk of memory, set a SEGV handler and commit more as needed. Basically how kernel thread stacks work. I've been meaning to do this but haven't gotten around to it yet.

Re: core.thread.Fiber --- runtime stack overflow unlike goroutines

2014-08-14 Thread Brad Anderson via Digitalmars-d-learn
On Thursday, 14 August 2014 at 07:46:29 UTC, Carl Sturtivant wrote: The default size of the runtime stack for a Fiber is 4*PAGESIZE which is very small, and a quick test shows that a Fiber suffers a stack overflow that doesn't lead to a clean termination when this limit is exceeded. This

Re: core.thread.Fiber --- runtime stack overflow unlike goroutines

2014-08-14 Thread Dicebot via Digitalmars-d-learn
On Thursday, 14 August 2014 at 18:52:00 UTC, Sean Kelly wrote: On 64 bit, reserve a huge chunk of memory, set a SEGV handler and commit more as needed. Basically how kernel thread stacks work. I've been meaning to do this but haven't gotten around to it yet. I think using some sort of