Yes, thank you for explaining... I believe it would fix the problem. num_bytes_decomp is over 16MB and Module._malloc(num_bytes_decomp) was causing the memory to grow
I'm doing it differently now with much smaller value (low hundreds KBs) for num_bytes_decomp so I'm not using automatic memory growth anymore and going with the default memory size Thanks a lot. Really appreciate it. On Mon, May 13, 2019 at 10:06 AM Alon Zakai <[email protected]> wrote: > That code might actually work even with growth - since I think you just > use dataHeap.byteOffset after the view might be invalid? I think the > byteOffset would be unchanged. > > But the safe way is to not create a view and refer to it after a call that > might grow memory. Creating all new views after any possible growth is the > safe thing. > > In this example, you can just save the byteOffset, which is just an > integer, to a local variable. > > > > On Thu, May 9, 2019 at 2:24 PM Marc Fawzi <[email protected]> wrote: > >> Oh, that does sound tricky, including in some single threaded scenarios, >> like the one below... >> >> How would you modify the code below to avoid landing in invalid memory? I >> can take a guess but ... still learning about the JS side of it >> >> function decompress(data / * Uint8Array */) { >> >> let dataPtr = Module._malloc(data.length); >> let dataHeap = new Uint8Array(Module.HEAPU8.buffer, dataPtr, >> data.length); >> dataHeap.set(data); >> >> let num_bytes_decomp= Module.C_getContentSize(dataHeap.byteOffset, >> data.length); >> let decompPtr = Module._malloc(num_bytes_decom); >> let decompHeap = new Uint8Array(Module.HEAPU8.buffer, decompPtr, >> num_bytes_decomp); >> >> let len = Module.C_decompress(decompHeap.byteOffset, >> num_bytes_decomp, dataHeap.byteOffset, data.length); >> >> let result = new Uint8Array(decompHeap.subarray(0, len)); >> >> Module._free(dataHeap.byteOffset); >> Module._free(decompHeap.byteOffset); >> >> return result >> } >> >> Thank you in advance >> >> On Thu, May 9, 2019 at 10:07 AM Alon Zakai <[email protected]> wrote: >> >>> Memory growth does mean you need to be careful when accessing the heap >>> from JavaScript. Emscripten will update the HEAPU8 etc. views, but if you >>> held on to references to the old views, or subarrays of them etc., those >>> would become invalid, and you'd silently miss updates to memory. Basically, >>> growth has the effect of creating replacements for all the JS memory views. >>> >>> (This is even trickier with multithreaded code using memory growth.) >>> >>> On Wed, May 8, 2019 at 10:34 AM Marc Fawzi <[email protected]> wrote: >>> >>>> Hi, >>>> >>>> Somehow when writing a large amount of data (over the 16MB default) to >>>> the Module heap (HEAPU8) I saw what could be a bug when ALLOW_MEMORY_GROWTH >>>> was set. Basically, when I print out the content I saw Emscripten’s own >>>> symbols, the C++ program symbols and a bunch of binary gibberish before the >>>> start of the the data I had written, as if my read offset was somehow >>>> smaller than my write offset. I’m working with someone else’ code and >>>> trying to understand what’s going on but when setting the memory manually >>>> at build time instead of allowing it to grow the code works just write and >>>> I get the exact data I write back when I read it. >>>> >>>> Any guides or blog posts out there for working with the heap? >>>> >>>> Thanks, >>>> >>>> -- >>>> You received this message because you are subscribed to the Google >>>> Groups "emscripten-discuss" group. >>>> To unsubscribe from this group and stop receiving emails from it, send >>>> an email to [email protected]. >>>> To view this discussion on the web visit >>>> https://groups.google.com/d/msgid/emscripten-discuss/CACioZivWDQ4HKHu6sdHq%2B%3Dyk2WX_GsiUcanTPDyNam1cWMmcUQ%40mail.gmail.com >>>> <https://groups.google.com/d/msgid/emscripten-discuss/CACioZivWDQ4HKHu6sdHq%2B%3Dyk2WX_GsiUcanTPDyNam1cWMmcUQ%40mail.gmail.com?utm_medium=email&utm_source=footer> >>>> . >>>> For more options, visit https://groups.google.com/d/optout. >>>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "emscripten-discuss" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to [email protected]. >>> To view this discussion on the web visit >>> https://groups.google.com/d/msgid/emscripten-discuss/CAEX4NpSDrHS89s-NxYStqn81RLD03Pg78Ns07QMc%3DMzdQzHKAw%40mail.gmail.com >>> <https://groups.google.com/d/msgid/emscripten-discuss/CAEX4NpSDrHS89s-NxYStqn81RLD03Pg78Ns07QMc%3DMzdQzHKAw%40mail.gmail.com?utm_medium=email&utm_source=footer> >>> . >>> For more options, visit https://groups.google.com/d/optout. >>> >> -- >> You received this message because you are subscribed to the Google Groups >> "emscripten-discuss" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/emscripten-discuss/CACioZisfumAkg%3Dk92L5zCmPgmnCzx5-TOr9rW6OHwwXA528_Jw%40mail.gmail.com >> <https://groups.google.com/d/msgid/emscripten-discuss/CACioZisfumAkg%3Dk92L5zCmPgmnCzx5-TOr9rW6OHwwXA528_Jw%40mail.gmail.com?utm_medium=email&utm_source=footer> >> . >> For more options, visit https://groups.google.com/d/optout. >> > -- > You received this message because you are subscribed to the Google Groups > "emscripten-discuss" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/emscripten-discuss/CAEX4NpREk6-PsvBv3rtOt3pW-MOgyD7VwXamnH%3DVEP37vwLtBA%40mail.gmail.com > <https://groups.google.com/d/msgid/emscripten-discuss/CAEX4NpREk6-PsvBv3rtOt3pW-MOgyD7VwXamnH%3DVEP37vwLtBA%40mail.gmail.com?utm_medium=email&utm_source=footer> > . > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "emscripten-discuss" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/emscripten-discuss/CACioZiu0GHU_YYxy%2BNchYYncgjq2T4SJp5hX%2BpNx4ScRbEwS4Q%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
