Thanks Alon, all - will try this out. From memory, by enabling multi-threading a lot of stuff was trying to use threads anyway. But this could well be an issue outside of Emscripten/emsdk (eg., `configure` scripts detecting threading support and enabling multi-threaded code paths). I'll double-check.
For us, much of the complexity of multi-threaded builds comes from the `-s PROXY_TO_PTHREAD` flag, which is necessary for anything asynchronous (we're running the Emscripten-generated code in a worker). On that note, I do hope that there will eventually be a threading 2.0 standard for WebAssembly that does no longer rely on browser APIs for threads and synchronisation (`SharedArrayBuffer`, `Atomics` APIs), but is more reminiscent of the fully self-contained threading support in (Portable) Native Client. Also, we've had linking errors as of late with emsdk 1.40.1 when compiling with `-s USE_PTHREADS` (and generally with emsdk 2.x). Probably a good opportunity to get to the bottom of this. Thank you again, Soeren On Wednesday, October 28, 2020 at 11:55:21 AM UTC+10 [email protected] wrote: > On Tue, Oct 27, 2020 at 8:29 AM Alon Zakai <[email protected]> wrote: > >> > As a hacky workaround, you could always provide your own memory by >> setting `Module['wasmMemory']` before loading the generated JS and then set >> the memory flag to shared using `wasm-dis + edit + wasm-as`. Obviously not >> ideal but maybe good for experimenting. >> >> I don't think that will work now that the Memory is not imported anymore >> in a normal build. You'd need to also mod the wasm to import the memory, >> and change the JS a little as well. >> > > We haven't made the switch yet I don't think (although I would like to > change that!). > > Its only STANDALONE_WASM I believe that creates and exports its memory > from the wasm module. The normal build still creates the memory object in > JS. > > >> >> But I agree building with pthreads enabled, but not actually using any >> pthreads APIs, seems like a good solution here. There may be a minor code >> size cost, is the only downside I can think of. >> >> On Tue, Oct 27, 2020 at 6:19 AM 'Sam Clegg' via emscripten-discuss < >> [email protected]> wrote: >> >>> This certainly seems like a reasonable setup. >>> >>> My first reaction would be why not just build with `USE_PTHREADS=1`? >>> This is the canonical way to tell emscripten to use shared memory. If >>> you don't actually start any threads I'm not sure what costs it has. What >>> "extra complexity of the WebAssembly threading model" are you referring to >>> here? >>> >>> If there are indeed significant costs to running single threaded >>> programs built with `USE_PTHREADS=1` then we can look into adding a >>> separate option for using shared memory, but I think it will be tricky to >>> persuade the linker wasm-ld to accept both shared memory, and object files >>> build for non-shared memory. >>> >>> As a hacky workaround, you could always provide your own memory by >>> setting `Module['wasmMemory']` before loading the generated JS and then set >>> the memory flag to shared using `wasm-dis + edit + wasm-as`. Obviously not >>> ideal but maybe good for experimenting. >>> >>> cheers, >>> sam >>> >>> On Tue, Oct 27, 2020 at 12:27 AM Soeren Balko <[email protected]> >>> wrote: >>> >>>> I've been running into an issue with Emscripten, where I cannot use a >>>> shared WebAssembly.Memory (ie., backed by a SharedArrayBuffer) in a >>>> single-threaded build. The rationale for this is to be able to use the >>>> Emscripten heap as a shared state with other workers / the UI thread. >>>> Specifically, it would allow us to directly copy data to the heap in other >>>> threads, avoiding to copy data. >>>> >>>> Our scenario looks like this: >>>> (1) We run a (single-threaded) WebAssembly module in a worker in a >>>> long-running synchronous call; >>>> (2) From within the synchronous call stack, the wasm module calls into >>>> JS to dispatch some work to the UI thread; >>>> (3) We currently COPY the data from the WebAssembly.Memory heap to a >>>> separate SharedArrayBuffer, which we then postMessage to the main thread. >>>> (4) The worker then uses a wait/notify pattern to block the Emscripten >>>> call stack until the UI thread has passed back a result on the >>>> SharedArrayBuffer >>>> (5) The UI thread can meanwhile perform some asynchronous work (eg., >>>> IndexedDB, etc.) and store the response in the SharedArrayBuffer, then >>>> notify the waiting worker. >>>> (6) The waiting worker then has to COPY the result from the >>>> SharedArrayBuffer back onto the Emscripten heap and return. >>>> >>>> In effect, we're performing two (expensive) copies. If the Emscripten >>>> heap was using a SharedArrayBuffer, we could trivially share that with the >>>> UI thread and merely pass offsets on the heap around. >>>> >>>> So in summary, are there any plans to supporting an SAB-backed >>>> WebAssembly.Memory for scenarios like these? I understand that a >>>> multi-threaded build could accomplish this. However, the extra complexity >>>> of the WebAssembly threading model (especially when running the >>>> WebAssembly >>>> module inside a worker) is not really acceptable in this case. >>>> >>>> Soeren >>>> >>>> -- >>>> You received this message because you are subscribed to the Google >>>> Groups "emscripten-discuss" group. >>>> To unsubscribe from this group and stop receiving emails from it, send >>>> an email to [email protected]. >>>> To view this discussion on the web visit >>>> https://groups.google.com/d/msgid/emscripten-discuss/283659cd-4e07-4913-a185-e4e9495f8d12n%40googlegroups.com >>>> >>>> <https://groups.google.com/d/msgid/emscripten-discuss/283659cd-4e07-4913-a185-e4e9495f8d12n%40googlegroups.com?utm_medium=email&utm_source=footer> >>>> . >>>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "emscripten-discuss" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to [email protected]. >>> To view this discussion on the web visit >>> https://groups.google.com/d/msgid/emscripten-discuss/CAL_va2_82PfYNs7%3DFHXfobPDQKt23Rg6nx3AhkhrFfbiJzTVoQ%40mail.gmail.com >>> >>> <https://groups.google.com/d/msgid/emscripten-discuss/CAL_va2_82PfYNs7%3DFHXfobPDQKt23Rg6nx3AhkhrFfbiJzTVoQ%40mail.gmail.com?utm_medium=email&utm_source=footer> >>> . >>> >> -- >> You received this message because you are subscribed to the Google Groups >> "emscripten-discuss" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> > To view this discussion on the web visit >> https://groups.google.com/d/msgid/emscripten-discuss/CAEX4NpQy6trmUWddtGCmZMNseKxBm%2B6xiVwSQpUzL2YagJgNPw%40mail.gmail.com >> >> <https://groups.google.com/d/msgid/emscripten-discuss/CAEX4NpQy6trmUWddtGCmZMNseKxBm%2B6xiVwSQpUzL2YagJgNPw%40mail.gmail.com?utm_medium=email&utm_source=footer> >> . >> > -- You received this message because you are subscribed to the Google Groups "emscripten-discuss" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/emscripten-discuss/7b20ad44-7efd-4ba4-9fbe-2300d771b536n%40googlegroups.com.
