Hey all, one of the particular points of attention we have been looking at is to ensure that it would be easy to write memory efficient applications with Emscripten. Runtime performance with asm.js, and even more so with wasm, is in the same ballpark with native, but with memory usage, it can be quite challenging to reach the same memory footprint that a native port has. A big reason for this is the in-memory filesystem MEMFS that Emscripten provides, which emulates synchronous filesystem access by storing all the files in memory all the time.
Also, since we've gained support for multithreading with SharedArrayBuffer, file I/O is something that we want to perform efficiently also in pthreads. In the current form, the MEMFS filesystem is unfortunately not shareable, but file I/O needs to be proxied between threads. We are working towards optimizing these aspects, and as a first step, I'd like to present a proposal for a new "Emscripten Fetch" API, which is a low level built-in runtime API for managing XHRs and IndexedDB operations from within Emscripten applications. The high level summary points for emscripten_fetch() are: - allows performing XHRs in a flexible manner, replacing the multiple wget variants we have (but does not deprecate) - allows interacting with IndexedDB storage for easy persistence strategies of XHRed data - multithread-aware, callable both in main thread and pthreads. (and works in non-multithreaded build modes as well) - defines a concept of asynchronous, synchronous and "waitable" operations for convenient management of when blocking vs nonblocking operations are needed. - a more finegrained control over XHRs/IndexedDB persistence of individual files, compared to the "full disk mount" -based MEMFS/IDBFS persistence behavior. Preliminary documentation, with code samples, is available at this point at http://clb.demon.fi/dump/emscripten_fetch/doc/docs/api_reference/fetch.html . If our contributors have a moment, feedback would be greatly appreciated. In particular, I'd like to nail the API structure down without missing use cases early on, so that we wouldn't risk having to do any kind of second iteration of emscripten_fetch_ex() immediately afterwards. The current implementation status is still in experimental phase. A lot of unit-test-like things already work, and the development branch lives in https://github.com/juj/emscripten/tree/emscripten_fetch A number of unit tests exists at https://github.com/juj/emscripten/tree/emscripten_fetch/tests/fetch, which can illustrate low level details of the work. A diff of the work is at https://github.com/kripken/emscripten/pull/4553 if anyone is interested in peeking under the hood. For the curious, a later planned follow-up step for this work is that there will be an "ASMFS" POSIX filesystem, which builds on top of emscripten_fetch() and its synchronous IndexedDB+XHR accesses to enable memory efficient multithreaded fopen()+fread()+fwrite() semantics that does not need proxying. Please let me know if you have questions or comments from this, either in here or as comments to the PR at https://github.com/kripken/emscripten/pull/4553. Thanks! Jukka -- You received this message because you are subscribed to the Google Groups "emscripten-discuss" group. To unsubscribe from this group and stop receiving emails from it, send an email to emscripten-discuss+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
