This looks like exactly what I need; I'll have to play with it to see if I can find a way to make a std::istream out of this, but if I can it'll be perfect. Thanks!
Richard > On Oct 13, 2015, at 3:18 PM, Alon Zakai <[email protected]> wrote: > > If performance is not a concern, then you could call into JS to reach each > byte, e.g. using EM_ASM. Or, if speed matters, you can read in chunks of 1K > or 1MB, writing them into a temporary spot in emscripten memory. This avoids > an extra copy of the whole 100MB file. e.g. > > void* temp = malloc(1024*1024); > for (int i = 0; i < numChunks; i++) { > EM_ASM_({ > HEAPU8.set(theBigBuffer.subarray($1 * 1024 * 1024, ($1 + 1) * 1024 * > 1024), $0); // copy 1MB from the middle of theBigBuffer into temp > }, temp, i); > // process that 1MB that was copied to temp > } > > (A more radical approach could use the new experimental split-memory mode, > https://github.com/kripken/emscripten/wiki/Split-Memory > <https://github.com/kripken/emscripten/wiki/Split-Memory> which actually > allows avoiding even those copies. However, it causes a very large slowdown > as a downside, so almost certainly not worth it.) > > > On Tue, Oct 13, 2015 at 9:52 AM, Richard Bateman <[email protected] > <mailto:[email protected]>> wrote: > Is it possible from emscripten code to read from an ArrayBuffer which is not > part of the heap? I have a very large file which I read from disk using > FileReader and would like to stream it as though from disk; I run out of > memory very quickly when copying it around before using it. > > If I could write a function which would be able to stream from the > ArrayBuffer then I could read a bit at a time rather than needing enough > memory to store an extra copy of my 100MB+ file. > > Thoughts? Suggestions? > > Thanks! > > Richard Bateman > GradeCam > > -- > You received this message because you are subscribed to the Google Groups > "emscripten-discuss" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected] > <mailto:emscripten-discuss%[email protected]>. > For more options, visit https://groups.google.com/d/optout > <https://groups.google.com/d/optout>. > > > -- > You received this message because you are subscribed to the Google Groups > "emscripten-discuss" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected] > <mailto:[email protected]>. > For more options, visit https://groups.google.com/d/optout > <https://groups.google.com/d/optout>. -- You received this message because you are subscribed to the Google Groups "emscripten-discuss" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
