Re: [rust-dev] Separating heaps from tasks
ARCs have their place, sure! But letting it leak isn't acceptable in my case. Instead, in my use case, no deletes until the whole heap is released makes way more sense (heaps are small, grow a bit, and get released). Since the lifetime of the object becomes == the lifetime of the heap, there's no issue with cycles. There's only an issue with multiple mutations, which like I said only needs a bit per pointer (and a non-atomic one at that as each heap is accessed by one thread - the only thing that gets sent between tasks is the whole heap!). So... different use cases, different solutions. ARC is a different trade-off. I guess the right thing to do would be to implement some sufficiently smart AppendOnlyHeapT pointer, but this seems hard to do (same heap can hold objects of multiple types, etc.) so for now I have some AlmostSafeHeapPointerT instead :-( Language support for heaps-separate-from-tasks would have solved it (and a bit more)... On Mon, Nov 4, 2013 at 8:32 AM, Daniel Micay danielmi...@gmail.com wrote: On Mon, Nov 4, 2013 at 1:29 AM, Oren Ben-Kiki o...@ben-kiki.org wrote: Even if RC allowed cycles (I don't quite see how...) the whole thing wouldn't be send-able, unless one uses ARC. But ARC has even more performance penalties than RC... And doing cycle-supporting ARC across tasks seems like pushing it - you might as well admit you are doing global GC in the 1st place. It can't support ownership cycles but it can certainly support cyclic links like `std::shared_ptr` + `std::weak_ptr` in C++. The performance penalty for supporting sends between tasks is atomic reference counting and the price for supporting weak pointers is an extra word per box. ___ Rust-dev mailing list Rust-dev@mozilla.org https://mail.mozilla.org/listinfo/rust-dev
Re: [rust-dev] Separating heaps from tasks
On 04/11/13 20:09, Oren Ben-Kiki wrote: ARCs have their place, sure! But letting it leak isn't acceptable in my case. Instead, in my use case, no deletes until the whole heap is released makes way more sense (heaps are small, grow a bit, and get released). Since the lifetime of the object becomes == the lifetime of the heap, there's no issue with cycles. There's only an issue with multiple mutations, which like I said only needs a bit per pointer (and a non-atomic one at that as each heap is accessed by one thread - the only thing that gets sent between tasks is the whole heap!). So... different use cases, different solutions. ARC is a different trade-off. I guess the right thing to do would be to implement some sufficiently smart AppendOnlyHeapT pointer, but this seems hard to do (same heap can hold objects of multiple types, etc.) so for now I have some AlmostSafeHeapPointerT instead :-( Language support for heaps-separate-from-tasks would have solved it (and a bit more)... Is this essentially an arena allocator where one can transfer the whole arena and all the objects allocated in it into another task? ___ Rust-dev mailing list Rust-dev@mozilla.org https://mail.mozilla.org/listinfo/rust-dev
Re: [rust-dev] Using libextra within libstd?
Is it possible to use stuff from libextra within libstd? It seems to me that it would set up a circular dependency Even if it was possible technically, probably not a good idea from a maintenance perspective to have such dependencies. On 4 November 2013 07:10, Martin DeMello martindeme...@gmail.com wrote: I've been looking at https://github.com/mozilla/rust/issues/6085 which seems like it should be fairly simple to fix, however, the proposed solution involves EnumSet from libextra. Is it possible to use stuff from libextra within libstd? It seems to me that it would set up a circular dependency, though that could just be my misunderstanding the rust compilation model. If it is possible, how would I do it? If not, what would be the proper fix for issue #6085? martin ___ Rust-dev mailing list Rust-dev@mozilla.org https://mail.mozilla.org/listinfo/rust-dev ___ Rust-dev mailing list Rust-dev@mozilla.org https://mail.mozilla.org/listinfo/rust-dev
Re: [rust-dev] Separating heaps from tasks
Essentially, yes. On Mon, Nov 4, 2013 at 11:59 AM, Huon Wilson dbau...@gmail.com wrote: On 04/11/13 20:09, Oren Ben-Kiki wrote: ARCs have their place, sure! But letting it leak isn't acceptable in my case. Instead, in my use case, no deletes until the whole heap is released makes way more sense (heaps are small, grow a bit, and get released). Since the lifetime of the object becomes == the lifetime of the heap, there's no issue with cycles. There's only an issue with multiple mutations, which like I said only needs a bit per pointer (and a non-atomic one at that as each heap is accessed by one thread - the only thing that gets sent between tasks is the whole heap!). So... different use cases, different solutions. ARC is a different trade-off. I guess the right thing to do would be to implement some sufficiently smart AppendOnlyHeapT pointer, but this seems hard to do (same heap can hold objects of multiple types, etc.) so for now I have some AlmostSafeHeapPointerT instead :-( Language support for heaps-separate-from-tasks would have solved it (and a bit more)... Is this essentially an arena allocator where one can transfer the whole arena and all the objects allocated in it into another task? ___ Rust-dev mailing list Rust-dev@mozilla.org https://mail.mozilla.org/listinfo/rust-dev ___ Rust-dev mailing list Rust-dev@mozilla.org https://mail.mozilla.org/listinfo/rust-dev
Re: [rust-dev] Separating heaps from tasks
This is not a complete answer to your question, but I have toyed with
the idea of an (unsafely implemented but with safe interface) message
allocation library that would partially address your use
case. Unfortunately I think that implementing it would require
higher-kinded types.
The basic would be an interface where you have an opaque message
that contains an arena and a root pointer (of some type `T`):
struct MessageT'a {
priv arena: ~Arena,
priv root: *mut T'static
}
You can create a message like so:
Message::new(|arena| {
let obj1 = arena.alloc(|| Leaf(...));
let obj2 = arena.alloc(|| Leaf(...));
arena.alloc(|| Root(obj1, obj2)) // return the root
});
You could open an existing message and revise its contents:
Message::edit(msg, |arena, root| {
...
root // return new root
})
These messages could be sent to other tasks, provided of course
that they do not access managed data and so forth.
The idea is to leverage the lifetime system to guarantee that pointers
allocated from the arena do not escape. I haven't thought too hard
about this, so there might be a hole, but I think it would work like
so:
implT'a:Isolate MessageT {
fn new(f: 'a |'a Arena| - 'a mut T'a {
let arena = ~Arena::new();
let root: *mut T'static = unsafe {
transmute(f(arena))
};
Message { arena: arena, root: root }
}
fn edit'a(m: mut Message'a,
f: 'b |'b Arena, 'b mut T'b| - 'b mut T'b) {
m.root = unsafe {
transmute(f(m.arena), transmute(m.root))
};
}
}
To address your use case, of course, we'd want to extend `arena` with
the ability to track the types of the memory it has allocated and
performance GC. Doing this while a message is being created or edited
would be challenging and would require hooks from the runtime to
obtain stack roots and so forth; interestingly, it'd be pretty trivial
to run the GC at deterministic times or (say) at the end of an editing
session. This might be enough for some uses cases.
Niko
[1]:
http://smallcultfollowing.com/babysteps/blog/2013/06/11/data-parallelism-in-rust/
On Mon, Nov 04, 2013 at 08:11:29AM +0200, Oren Ben-Kiki wrote:
I am toying with a non-trivial Rust project to get a feel for the language.
There's a pattern I keep seeing in my code which isn't easy to express in
Rust. I wonder what the right thing to do is here.
The pattern is as follows. I have some container, which contains some
components of different types. The container as a whole is send-able . The
components form a complex graph (with cycles).
What I'd like to do is something like this:
- Declare a pool of objects of some types, which is held by the container.
- Declare pointer-to-object whose scope is the container; that is, the
lifetime of the pointer is the lifetime of the container. The pointer can
be freely passed around, cloned, etc. but (for mutable objects), only one
mutable access is allowed at a time.
This calls for something between GC pointers and RC pointers. GC is out,
because it isn't send-able. RC is out, because it doesn't allow for loops.
So right now I use explicit pools and a semi-safe (that is, unsafe...)
smart pointer type. And I don't support dropping objects until the whole
container is done (which is OK in my specific app but isn't really a good
solution).
Ideally what I'd like to see is separating heaps from tasks. That is,
suppose that GC pointers had a heap attribute (like borrowed pointers have
a lifetime attribute). By default, each task has a heap, but it is also
possible to define additional heaps (like we have the static lifetime and
can also define additional lifetimes).
So, the container could hold a heap and then many components with
heap-scoped pointers. The whole thing is send-able and GC is done in the
scope of each heap on its own (like today).
There are implications on the type system (no mixing pointers between
different heaps, unless the heaps are nested) - this seems very similar to
the lifetimes type checking...
Overall this seems very symmetrical with lifetimes. Basically, lifetimes ==
static (compile-time computable) free/malloc; heaps == dynamic (run-time
computable) free/malloc.
One interesting pattern allowed by this is ad-hoc actors (there are others
of course). Currently, if one wants to write actor-style code, one ties in
the GC pointers to one heap of one actor, which means one forces the
parallelization policy to one task per actor. One could argue that the
run-time should be good enough that any aggregation of actor threads to OS
threads would be done optimally (which is a good goal); but in some apps,
to get good performance one would like to control this. If we could
separate heaps from tasks, we could spawn fewer tasks (roughly the number
of OS threads) and use
Re: [rust-dev] Using libextra within libstd?
On 11/03/2013 11:10 PM, Martin DeMello wrote: I've been looking at https://github.com/mozilla/rust/issues/6085 which seems like it should be fairly simple to fix, however, the proposed solution involves EnumSet from libextra. Is it possible to use stuff from libextra within libstd? It seems to me that it would set up a circular dependency, though that could just be my misunderstanding the rust compilation model. If it is possible, how would I do it? If not, what would be the proper fix for issue #6085? As others mentioned it's not generally possible, but just for curiosity's sake I'll point out that when running tests std *does* link to and use features from libextra. It's mind-bending and bad. When we decide that std absolutely can't live without features from extra, then those features get promoted to std. The bar is pretty high though. ___ Rust-dev mailing list Rust-dev@mozilla.org https://mail.mozilla.org/listinfo/rust-dev
Re: [rust-dev] Using libextra within libstd?
hello I'm new in the rust dev list, so sorry if the question has already been rised. But what is the reason to have 2 libraries std/extra? why not gathering all in a single std library? Thanks G. - Gaetan 2013/11/4 Brian Anderson bander...@mozilla.com On 11/03/2013 11:10 PM, Martin DeMello wrote: I've been looking at https://github.com/mozilla/rust/issues/6085 which seems like it should be fairly simple to fix, however, the proposed solution involves EnumSet from libextra. Is it possible to use stuff from libextra within libstd? It seems to me that it would set up a circular dependency, though that could just be my misunderstanding the rust compilation model. If it is possible, how would I do it? If not, what would be the proper fix for issue #6085? As others mentioned it's not generally possible, but just for curiosity's sake I'll point out that when running tests std *does* link to and use features from libextra. It's mind-bending and bad. When we decide that std absolutely can't live without features from extra, then those features get promoted to std. The bar is pretty high though. ___ Rust-dev mailing list Rust-dev@mozilla.org https://mail.mozilla.org/listinfo/rust-dev ___ Rust-dev mailing list Rust-dev@mozilla.org https://mail.mozilla.org/listinfo/rust-dev
Re: [rust-dev] Using libextra within libstd?
On 11/04/2013 11:00 AM, Gaetan wrote: hello I'm new in the rust dev list, so sorry if the question has already been rised. But what is the reason to have 2 libraries std/extra? why not gathering all in a single std library? There are a few reasons for the split between std and extra. In general, we think the correct strategy for organizing Rust code is to use many small crates; one of the main reasons for this is to avoid large compilation times, but there are arguable modularity benefits as well. Personally I'd like to avoid Java's situation where there is one very large monolithic standard library that we have to live with in its entirety forever. The 'vision' currently is to have a rich ecosystem of officially supported packages in addition to the monolithic standard library. Under this scheme libextra will not exist at all, and will instead be broken up into a bunch of individual packages in their own repositories. Packages can be developed and stabilized and deprecated by the larger community on their own schedules, and those that fall out of use can disappear gracefully. The standard library though is monolithic because it has many complex interdependencies and wasn't developed with any intent to be more modular. In my opinion the definition of std is something like 'all the abstractions that depend on some interface to the compiler (this includes primitive types, the runtime, atomics), plus those that are so common that most moderate-sized software will need them (various data structures, algorithms and OS interfaces)'. I don't actually recall the original motivation for the split, but this is my current thinking. ___ Rust-dev mailing list Rust-dev@mozilla.org https://mail.mozilla.org/listinfo/rust-dev
Re: [rust-dev] Using libextra within libstd?
How does this work without getting into a dependency loop at build time? martin On Mon, Nov 4, 2013 at 10:55 AM, Brian Anderson bander...@mozilla.com wrote: On 11/03/2013 11:10 PM, Martin DeMello wrote: I've been looking at https://github.com/mozilla/rust/issues/6085 which seems like it should be fairly simple to fix, however, the proposed solution involves EnumSet from libextra. Is it possible to use stuff from libextra within libstd? It seems to me that it would set up a circular dependency, though that could just be my misunderstanding the rust compilation model. If it is possible, how would I do it? If not, what would be the proper fix for issue #6085? As others mentioned it's not generally possible, but just for curiosity's sake I'll point out that when running tests std *does* link to and use features from libextra. It's mind-bending and bad. When we decide that std absolutely can't live without features from extra, then those features get promoted to std. The bar is pretty high though. ___ Rust-dev mailing list Rust-dev@mozilla.org https://mail.mozilla.org/listinfo/rust-dev ___ Rust-dev mailing list Rust-dev@mozilla.org https://mail.mozilla.org/listinfo/rust-dev
Re: [rust-dev] Using libextra within libstd?
ok I understand, it will be like python which I really like the modularity. I'm beginning to play a little with std::os as my first rust programming, trying to add missing methods, etc, and I think this should quite strategic to move to libextra and then to be splitted appart, this would solve this cyclic dependency. What do you think about it? - Gaetan 2013/11/4 Martin DeMello martindeme...@gmail.com How does this work without getting into a dependency loop at build time? martin On Mon, Nov 4, 2013 at 10:55 AM, Brian Anderson bander...@mozilla.com wrote: On 11/03/2013 11:10 PM, Martin DeMello wrote: I've been looking at https://github.com/mozilla/rust/issues/6085 which seems like it should be fairly simple to fix, however, the proposed solution involves EnumSet from libextra. Is it possible to use stuff from libextra within libstd? It seems to me that it would set up a circular dependency, though that could just be my misunderstanding the rust compilation model. If it is possible, how would I do it? If not, what would be the proper fix for issue #6085? As others mentioned it's not generally possible, but just for curiosity's sake I'll point out that when running tests std *does* link to and use features from libextra. It's mind-bending and bad. When we decide that std absolutely can't live without features from extra, then those features get promoted to std. The bar is pretty high though. ___ Rust-dev mailing list Rust-dev@mozilla.org https://mail.mozilla.org/listinfo/rust-dev ___ Rust-dev mailing list Rust-dev@mozilla.org https://mail.mozilla.org/listinfo/rust-dev ___ Rust-dev mailing list Rust-dev@mozilla.org https://mail.mozilla.org/listinfo/rust-dev
Re: [rust-dev] Using libextra within libstd?
On 11/04/2013 02:39 PM, Martin DeMello wrote: How does this work without getting into a dependency loop at build time? The std under test is not *actually* the standard library as seen by compiler and runtime; as far as the compiler is concerned it is just some library compiled with --test, that links to libextra and libstd (so when testing std there are two different copies of it in memory.. This has some complications since the standard library has some functionality (lang-items) that must not be redefined, so std has a few strategic 'cfg' attributes thrown in to avoid that. For the most part this set up works just fine, but occasionally can result in some very hard to understand errors, especially if you run tests of the runtime without first recompiling the standard library (resulting in unsafe interoperation of incompatible type definitions). martin On Mon, Nov 4, 2013 at 10:55 AM, Brian Anderson bander...@mozilla.com wrote: On 11/03/2013 11:10 PM, Martin DeMello wrote: I've been looking at https://github.com/mozilla/rust/issues/6085 which seems like it should be fairly simple to fix, however, the proposed solution involves EnumSet from libextra. Is it possible to use stuff from libextra within libstd? It seems to me that it would set up a circular dependency, though that could just be my misunderstanding the rust compilation model. If it is possible, how would I do it? If not, what would be the proper fix for issue #6085? As others mentioned it's not generally possible, but just for curiosity's sake I'll point out that when running tests std *does* link to and use features from libextra. It's mind-bending and bad. When we decide that std absolutely can't live without features from extra, then those features get promoted to std. The bar is pretty high though. ___ Rust-dev mailing list Rust-dev@mozilla.org https://mail.mozilla.org/listinfo/rust-dev ___ Rust-dev mailing list Rust-dev@mozilla.org https://mail.mozilla.org/listinfo/rust-dev
Re: [rust-dev] Using libextra within libstd?
On 11/04/2013 03:09 PM, Gaetan wrote: ok I understand, it will be like python which I really like the modularity. I'm beginning to play a little with std::os as my first rust programming, trying to add missing methods, etc, and I think this should quite strategic to move to libextra and then to be splitted appart, this would solve this cyclic dependency. What do you think about it? Things like 'mkdir' are pretty important and are increasingly tied to our I/O subsystem (it just moved into std::rt::io::fs), so I don't think it should move out of standard. Since this issue of how to represent OR-able flags is pretty common, the consensus seems to be to clean up EnumSet and move it to std. If you're interested in tackling it I opened an issue specifically about the move https://github.com/mozilla/rust/issues/10272. - Gaetan 2013/11/4 Martin DeMello martindeme...@gmail.com mailto:martindeme...@gmail.com How does this work without getting into a dependency loop at build time? martin On Mon, Nov 4, 2013 at 10:55 AM, Brian Anderson bander...@mozilla.com mailto:bander...@mozilla.com wrote: On 11/03/2013 11:10 PM, Martin DeMello wrote: I've been looking at https://github.com/mozilla/rust/issues/6085 which seems like it should be fairly simple to fix, however, the proposed solution involves EnumSet from libextra. Is it possible to use stuff from libextra within libstd? It seems to me that it would set up a circular dependency, though that could just be my misunderstanding the rust compilation model. If it is possible, how would I do it? If not, what would be the proper fix for issue #6085? As others mentioned it's not generally possible, but just for curiosity's sake I'll point out that when running tests std *does* link to and use features from libextra. It's mind-bending and bad. When we decide that std absolutely can't live without features from extra, then those features get promoted to std. The bar is pretty high though. ___ Rust-dev mailing list Rust-dev@mozilla.org mailto:Rust-dev@mozilla.org https://mail.mozilla.org/listinfo/rust-dev ___ Rust-dev mailing list Rust-dev@mozilla.org mailto:Rust-dev@mozilla.org https://mail.mozilla.org/listinfo/rust-dev ___ Rust-dev mailing list Rust-dev@mozilla.org https://mail.mozilla.org/listinfo/rust-dev
Re: [rust-dev] Using libextra within libstd?
okay, i figure niko might want to do this himself, but i've marked my interest on the bug if he doesn't. martin On Mon, Nov 4, 2013 at 4:42 PM, Brian Anderson bander...@mozilla.com wrote: On 11/04/2013 03:09 PM, Gaetan wrote: ok I understand, it will be like python which I really like the modularity. I'm beginning to play a little with std::os as my first rust programming, trying to add missing methods, etc, and I think this should quite strategic to move to libextra and then to be splitted appart, this would solve this cyclic dependency. What do you think about it? Things like 'mkdir' are pretty important and are increasingly tied to our I/O subsystem (it just moved into std::rt::io::fs), so I don't think it should move out of standard. Since this issue of how to represent OR-able flags is pretty common, the consensus seems to be to clean up EnumSet and move it to std. If you're interested in tackling it I opened an issue specifically about the move https://github.com/mozilla/rust/issues/10272. - Gaetan 2013/11/4 Martin DeMello martindeme...@gmail.com How does this work without getting into a dependency loop at build time? martin On Mon, Nov 4, 2013 at 10:55 AM, Brian Anderson bander...@mozilla.com wrote: On 11/03/2013 11:10 PM, Martin DeMello wrote: I've been looking at https://github.com/mozilla/rust/issues/6085 which seems like it should be fairly simple to fix, however, the proposed solution involves EnumSet from libextra. Is it possible to use stuff from libextra within libstd? It seems to me that it would set up a circular dependency, though that could just be my misunderstanding the rust compilation model. If it is possible, how would I do it? If not, what would be the proper fix for issue #6085? As others mentioned it's not generally possible, but just for curiosity's sake I'll point out that when running tests std *does* link to and use features from libextra. It's mind-bending and bad. When we decide that std absolutely can't live without features from extra, then those features get promoted to std. The bar is pretty high though. ___ Rust-dev mailing list Rust-dev@mozilla.org https://mail.mozilla.org/listinfo/rust-dev ___ Rust-dev mailing list Rust-dev@mozilla.org https://mail.mozilla.org/listinfo/rust-dev ___ Rust-dev mailing list Rust-dev@mozilla.org https://mail.mozilla.org/listinfo/rust-dev
Re: [rust-dev] Using libextra within libstd?
On 11/04/2013 04:59 PM, Martin DeMello wrote: okay, i figure niko might want to do this himself, but i've marked my interest on the bug if he doesn't. Ask niko for advice about the design, but he will be *happy* to let you do it, guaranteed. ___ Rust-dev mailing list Rust-dev@mozilla.org https://mail.mozilla.org/listinfo/rust-dev
[rust-dev] Abandoning segmented stacks in Rust
Greetings you delightful Rustic people, Sound the death knell for segmented stacks. The new runtime does not implement segmented stacks and it never will. This decision has been brewing for some time, and now I'd like to make it official and lay out the reasoning for posterity. To recap, segmented stacks are a strategy for growing stacks incrementally, starting from a single small stack segment, and appending further segments as needed in a linked list as the stack grows. This allows threads to be very compact so that many can be created concurrently. The previous Rust runtime did implement segmented stacks, and that experience makes us believe that the performance tradeoffs required are not compatible with Rust's goals as a high performance systems language. The performance problems associated with segmented stacks can be summerized as: switching stacks has a cost. Though that cost can be minimized through extensive tuning and micro-optimization (much of which we admittedly did not pursue), it will never be free, and we've concluded that the effort and complexity of continuing down that route is not justified. We've seen the problem manifest primarily in three areas: * stack thrashing - when out of stack a function call will force an allocation of a new segment, which is later freed upon return. This is expensive even when the new stack segmented is cached. The result is unpredictable and severe drops in performance whenever a stack boundary happens to fall inside a tight loop. ([Similar concerns][1] are pushing Go away from a segmented stack scheme as well - they call it the hot split problem). [1]: https://docs.google.com/document/d/1wAaf1rYoM4S4gtnPh0zOlGzWtrZFQ5suE8qr2sD8uWQ/pub * FFI - foreign code typically expects to have large stacks, so using the FFI often requires switching stacks. Avoiding this overhead would require an elaborate and inherently unsafe system of annotation (#8822), increasing a burden on the FFI interface. * LLVM libc optimizations and intrinsic fallbacks - LLVM will transform calls to some libc functions into intrinsics and some intrinsics into runtime function calls, for reasons of performance and platform compatibility. Obvious solutions to making these compatible with segmented stacks impose a high minimum stack requirement, partially defeating the point of a segmented stack. Instead of segmented stacks we're going to rely on the OS and MMU to help us map pages lazily. Although the details aren't clear yet, I expect that on 64-bit platforms the number of concurrent tasks will be comparable to using segmented stacks. On 32-bit platforms, with their limited address space, the situation will not be as good, but this is a calculated risk that we can live without the same amount of concurrency there. Regards, Brian ___ Rust-dev mailing list Rust-dev@mozilla.org https://mail.mozilla.org/listinfo/rust-dev
Re: [rust-dev] Abandoning segmented stacks in Rust
Bill, memory is cheap. On Mon, Nov 4, 2013 at 9:50 PM, Bill Myers bill_my...@outlook.com wrote: The advantage of segmented stacks is that blocked tasks only take up as much memory as they actually need to store state, so that for instance a network server can use a task for each connection, and still only use, say, 64 bytes per connection if that's possible instead of the number of stack pages that got allocated for previous computation (assuming an extreme version that allocates a stack segment on every call). However, there is another approach that can replace segmented stacks for that purpose, namely having the compiler automatically transform blocking functions to instead return a future (with limited lifetime). This means that segmented allocation only happens for functions that indirectly perform I/O and only allocates the exact amount of memory needed to retain state that must persistent across the blocking I/O operation, while other functions execute normally using traditional stacks. The simplest example of this feature is async/await in C# 5, and Scala has a delimited continuation passing transformation that can be used to do the same thing. Has this been considered for Rust? ___ Rust-dev mailing list Rust-dev@mozilla.org https://mail.mozilla.org/listinfo/rust-dev -- -Thad +ThadGuidry https://www.google.com/+ThadGuidry Thad on LinkedIn http://www.linkedin.com/in/thadguidry/ ___ Rust-dev mailing list Rust-dev@mozilla.org https://mail.mozilla.org/listinfo/rust-dev
Re: [rust-dev] Abandoning segmented stacks in Rust
Note that as memory becomes cheaper and larger there will be more pressure on 64-bit OS-es to switch to large pages; the number of pages needed to map several GBs of memory today is already getting out of hand, causing TLB misses to become a performance issue in some cases - imagine a system with 0.xTBs of memory and it becomes ludicrous. So playing tricks with MMU and lazy page loading may not work as well as it does with today's the small 4K page size. Of course, Rust is hardly the only platform that would be affected :-) and ideally, it would be possible to have more flexibility than today in choosing which page sizes are used where in the program's address space... but it remains to be seen how exactly this would play out. Just a point to keep in mind... On Tue, Nov 5, 2013 at 4:21 AM, Brian Anderson bander...@mozilla.comwrote: Instead of segmented stacks we're going to rely on the OS and MMU to help us map pages lazily. Although the details aren't clear yet, I expect that on 64-bit platforms the number of concurrent tasks will be comparable to using segmented stacks. On 32-bit platforms, with their limited address space, the situation will not be as good, but this is a calculated risk that we can live without the same amount of concurrency there. ___ Rust-dev mailing list Rust-dev@mozilla.org https://mail.mozilla.org/listinfo/rust-dev
Re: [rust-dev] Abandoning segmented stacks in Rust
Having memory is cheap, accessing it isn't.
On Monday, November 4, 2013, Thad Guidry wrote:
Bill, memory is cheap.
On Mon, Nov 4, 2013 at 9:50 PM, Bill Myers
bill_my...@outlook.comjavascript:_e({}, 'cvml', 'bill_my...@outlook.com');
wrote:
The advantage of segmented stacks is that blocked tasks only take up as
much memory as they actually need to store state, so that for instance a
network server can use a task for each connection, and still only use, say,
64 bytes per connection if that's possible instead of the number of stack
pages that got allocated for previous computation (assuming an extreme
version that allocates a stack segment on every call).
However, there is another approach that can replace segmented stacks for
that purpose, namely having the compiler automatically transform blocking
functions to instead return a future (with limited lifetime).
This means that segmented allocation only happens for functions that
indirectly perform I/O and only allocates the exact amount of memory needed
to retain state that must persistent across the blocking I/O operation,
while other functions execute normally using traditional stacks.
The simplest example of this feature is async/await in C# 5, and Scala
has a delimited continuation passing transformation that can be used to do
the same thing.
Has this been considered for Rust?
___
Rust-dev mailing list
Rust-dev@mozilla.org javascript:_e({}, 'cvml', 'Rust-dev@mozilla.org');
https://mail.mozilla.org/listinfo/rust-dev
--
-Thad
+ThadGuidry https://www.google.com/+ThadGuidry
Thad on LinkedIn http://www.linkedin.com/in/thadguidry/
___
Rust-dev mailing list
Rust-dev@mozilla.org
https://mail.mozilla.org/listinfo/rust-dev
