One of the few limitations I had using Emscripten was actually down to the
browser environment not allowing shared state between web workers. With
so-called *transferable objects* being able to make extremely fast copies,
and Emscripten providing a basic API to wrap up delivering and receiving
byte arrays, this was somewhat mitigated.
However, in my particular case, the objects in question were quite
heavyweight, and serialising/deserialising was really slow, even in a
native environment. The other option Alon suggested at the time (about a
year ago now) was to use bump or offset allocators - but the problem with
that was having to patch pretty complex and heavily templated third party
code, with little prospect of it being merged upstream.
I was thinking about this recently again, and wondered if it would be
possible to optionally reserve an area in the Emscripten heap to get around
the dual issues of a) pointer offsets, and b) not being able/willing to
change third party code. I'd imagine it working like this:
- All Emscripten runtimes allocate a fixed chunk of memory low down in
the heap (of a configurable size). This area of the heap would have to be
allocated before anything else in the heap, to ensure it was at the *same*
start location in all Emscripten runtime instances.
- I could then write objects to this specific area either by:
- A) Turning on a global flag (when needed) that would temporarily
override the global new operator, then use a copy constructor on the
objects I was interested in (such that all objects resulting from 'new'
invocations internal to the third party code were placed in the reserved
area), then turn off the global flag.
- B) Even better, be able at Emscripten compile time to somehow
specify classnames that should always be allocated to the reserved space
-
but I'm not entirely sure if that's possible.
- When you're ready, the main Emscripten runtime sends a message to the
web worker with a copy of the whole reserved area, and the receiving web
worker runtime copies it into its own reserved area, which is of the same
size and location.
- The web worker Emscripten runtime could then use objects in that
reserved area, with all internal pointers of the object not having any
offset issues.
- Sending results the other way would be similar - the only difference
being that the receiving main runtime would have to copy the result out of
the shared memory area (again with a copy constructor), to free it up
straight away for other web workers needing to return results.
An example of using this would be:
// If we've gone with option b), the pointer to this object, and all its
internal pointers, will automatically be in the reserved area
BigObject *a = new BigObject();
// If we went with option a) we'd have to do something like
turn_on_reserved_new();
// *copy will now be in the reserved area
BigObject *copy = new BigObject(a);
turn_off_reservered_new();
// Send the reserved area to the worker
sendMessage(reserved_area[], reserved_area_size);
// Now the web worker can receive jobs that simply refer to pointers in the
reserved area.
Not sure if this would all be feasible, or in any case whether this is a
edge case that wouldn't be worth such effort?
--
You received this message because you are subscribed to the Google Groups
"emscripten-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
For more options, visit https://groups.google.com/d/optout.
