As someone trying to build code that has emscripten send/recv messages to/from web workers, I'd love to see some experimentation like this.
In the case of network/mapped file simulation the system could even be much simpler: compile time setting for # and size of reserved buffers, then runtime calls for isolate_alloc, isolate_free, and isolate_send/etc to access them. Being all fixed size and of a special type should make things a much easier, as I'm pretty sure to implement something like this one would have to have a pointer type with an attribute that emscripten/LLVM could use to handle the memory accesses differently. Like, you couldn't just cast this special memory to a char* (or BigObject* or whatever) and party on it, but instead have to use an isolate_ptr<char> (or some other fancy way of doing things). I've been wanting this, but haven't had time to try hacking it in. Something like this would allow for some high performance/zero garbage data transfer, though, and be pretty awesome for those trying to write multithreaded apps. On Sat, Apr 26, 2014 at 7:25 PM, Marcos Scriven <[email protected]> wrote: > One of the few limitations I had using Emscripten was actually down to the > browser environment not allowing shared state between web workers. With > so-called *transferable objects* being able to make extremely fast > copies, and Emscripten providing a basic API to wrap up delivering and > receiving byte arrays, this was somewhat mitigated. > > However, in my particular case, the objects in question were quite > heavyweight, and serialising/deserialising was really slow, even in a > native environment. The other option Alon suggested at the time (about a > year ago now) was to use bump or offset allocators - but the problem with > that was having to patch pretty complex and heavily templated third party > code, with little prospect of it being merged upstream. > > I was thinking about this recently again, and wondered if it would be > possible to optionally reserve an area in the Emscripten heap to get around > the dual issues of a) pointer offsets, and b) not being able/willing to > change third party code. I'd imagine it working like this: > > - All Emscripten runtimes allocate a fixed chunk of memory low down in > the heap (of a configurable size). This area of the heap would have to be > allocated before anything else in the heap, to ensure it was at the *same* > start location in all Emscripten runtime instances. > - I could then write objects to this specific area either by: > - A) Turning on a global flag (when needed) that would temporarily > override the global new operator, then use a copy constructor on the > objects I was interested in (such that all objects resulting from 'new' > invocations internal to the third party code were placed in the reserved > area), then turn off the global flag. > - B) Even better, be able at Emscripten compile time to somehow > specify classnames that should always be allocated to the reserved > space - > but I'm not entirely sure if that's possible. > - When you're ready, the main Emscripten runtime sends a message to > the web worker with a copy of the whole reserved area, and the receiving > web worker runtime copies it into its own reserved area, which is of the > same size and location. > - The web worker Emscripten runtime could then use objects in that > reserved area, with all internal pointers of the object not having any > offset issues. > - Sending results the other way would be similar - the only difference > being that the receiving main runtime would have to copy the result out of > the shared memory area (again with a copy constructor), to free it up > straight away for other web workers needing to return results. > > An example of using this would be: > > // If we've gone with option b), the pointer to this object, and all its > internal pointers, will automatically be in the reserved area > BigObject *a = new BigObject(); > > // If we went with option a) we'd have to do something like > turn_on_reserved_new(); > // *copy will now be in the reserved area > BigObject *copy = new BigObject(a); > turn_off_reservered_new(); > > // Send the reserved area to the worker > sendMessage(reserved_area[], reserved_area_size); > > // Now the web worker can receive jobs that simply refer to pointers in > the reserved area. > > > > Not sure if this would all be feasible, or in any case whether this is a > edge case that wouldn't be worth such effort? > > > > -- > You received this message because you are subscribed to the Google Groups > "emscripten-discuss" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "emscripten-discuss" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
