Not sure if it's relevant, but LLVM has support for "address spaces", which tags pointers basically. In theory something like that might help here, but I'm not sure how.
- Alon On Mon, Apr 28, 2014 at 7:32 AM, 'Ben Vanik' via emscripten-discuss < [email protected]> wrote: > As someone trying to build code that has emscripten send/recv messages > to/from web workers, I'd love to see some experimentation like this. > > In the case of network/mapped file simulation the system could even be > much simpler: compile time setting for # and size of reserved buffers, then > runtime calls for isolate_alloc, isolate_free, and isolate_send/etc to > access them. Being all fixed size and of a special type should make things > a much easier, as I'm pretty sure to implement something like this one > would have to have a pointer type with an attribute that emscripten/LLVM > could use to handle the memory accesses differently. Like, you couldn't > just cast this special memory to a char* (or BigObject* or whatever) and > party on it, but instead have to use an isolate_ptr<char> (or some other > fancy way of doing things). > > I've been wanting this, but haven't had time to try hacking it in. > Something like this would allow for some high performance/zero garbage data > transfer, though, and be pretty awesome for those trying to write > multithreaded apps. > > > On Sat, Apr 26, 2014 at 7:25 PM, Marcos Scriven <[email protected]>wrote: > >> One of the few limitations I had using Emscripten was actually down to >> the browser environment not allowing shared state between web workers. With >> so-called *transferable objects* being able to make extremely fast >> copies, and Emscripten providing a basic API to wrap up delivering and >> receiving byte arrays, this was somewhat mitigated. >> >> However, in my particular case, the objects in question were quite >> heavyweight, and serialising/deserialising was really slow, even in a >> native environment. The other option Alon suggested at the time (about a >> year ago now) was to use bump or offset allocators - but the problem with >> that was having to patch pretty complex and heavily templated third party >> code, with little prospect of it being merged upstream. >> >> I was thinking about this recently again, and wondered if it would be >> possible to optionally reserve an area in the Emscripten heap to get around >> the dual issues of a) pointer offsets, and b) not being able/willing to >> change third party code. I'd imagine it working like this: >> >> - All Emscripten runtimes allocate a fixed chunk of memory low down >> in the heap (of a configurable size). This area of the heap would have to >> be allocated before anything else in the heap, to ensure it was at the >> *same* start location in all Emscripten runtime instances. >> - I could then write objects to this specific area either by: >> - A) Turning on a global flag (when needed) that would temporarily >> override the global new operator, then use a copy constructor on the >> objects I was interested in (such that all objects resulting from 'new' >> invocations internal to the third party code were placed in the >> reserved >> area), then turn off the global flag. >> - B) Even better, be able at Emscripten compile time to somehow >> specify classnames that should always be allocated to the reserved >> space - >> but I'm not entirely sure if that's possible. >> - When you're ready, the main Emscripten runtime sends a message to >> the web worker with a copy of the whole reserved area, and the receiving >> web worker runtime copies it into its own reserved area, which is of the >> same size and location. >> - The web worker Emscripten runtime could then use objects in that >> reserved area, with all internal pointers of the object not having any >> offset issues. >> - Sending results the other way would be similar - the only >> difference being that the receiving main runtime would have to copy the >> result out of the shared memory area (again with a copy constructor), to >> free it up straight away for other web workers needing to return results. >> >> An example of using this would be: >> >> // If we've gone with option b), the pointer to this object, and all its >> internal pointers, will automatically be in the reserved area >> BigObject *a = new BigObject(); >> >> // If we went with option a) we'd have to do something like >> turn_on_reserved_new(); >> // *copy will now be in the reserved area >> BigObject *copy = new BigObject(a); >> turn_off_reservered_new(); >> >> // Send the reserved area to the worker >> sendMessage(reserved_area[], reserved_area_size); >> >> // Now the web worker can receive jobs that simply refer to pointers in >> the reserved area. >> >> >> >> Not sure if this would all be feasible, or in any case whether this is a >> edge case that wouldn't be worth such effort? >> >> >> >> -- >> You received this message because you are subscribed to the Google Groups >> "emscripten-discuss" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> For more options, visit https://groups.google.com/d/optout. >> > > -- > You received this message because you are subscribed to the Google Groups > "emscripten-discuss" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "emscripten-discuss" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
