Thanks Alon! That explains it! Yeah I should have thought a little deeper. I am just posting my follow up question in case you did not get a chance to look at it.
One follow up question. May be a dumb one. What could be the potential problems with ALLOW_MEMORY_GROWTH missing in PThreads mode? I see that when the Wasm is instantiated, the overall memory that the Chrome tab was taking was similar to the one taken by the WASM built with ALLOW_MEMORY_GROWTH. Is it that, we will not be able to instantiate WASM on low end devices if built with ALLOW_MEMORY_GROWTH=0? Regards, Prashanth Nethi On Thursday, August 27, 2020 at 2:37:07 AM UTC+5:30 [email protected] wrote: > My guess is that's because of the behavior of std::vector and how it > resizes. Over those appends it will malloc and free repeatedly and that may > cause fragmentation that prevents a final larger size, which must be a > single contiguous region. The second version allocates many smaller ones, > not a single contiguous region. > > - Alon > > > On Tue, Aug 25, 2020 at 11:24 PM Prashanth Nethi <[email protected]> > wrote: > >> Thanks Alon! So here is something very weird. I could get the memory >> usage go all the way to 2GB when I changed my testing code. This was my >> original test code. So basically I was just adding elements to std::vector >> infinitely. >> >> class TestClass{ >> private: >> int t = 0; >> }; >> >> struct Data { >> int t; >> TestClass obj; >> }; >> >> typedef std::vector<Data> Vec; >> >> Vec someVec; >> >> using namespace std; >> >> int main() { >> printf("hello, world!\n"); >> >> while(1){ >> Data data; >> someVec.push_back(data); >> } >> >> return 0; >> } >> >> With this code, the WASM memory was going all the way to 1GB. >> >> But when I changed the code to this, where I am writing some value after >> acquiring memory, then I am able to see the memory usage go all the way up >> to 2 GB. Could this be a bug? I am on emscripten 2.0. >> >> >> int main() { >> printf("hello, world!\n"); >> char *p = nullptr; >> int byteSize = 50 * 1024 * 1024; >> while(1){ >> p = new char(byteSize); >> p[byteSize] = 20; >> } >> return 0; >> } >> >> Also It is very encouraging to see that 4GB is considered for PThreads as >> well! Thanks. >> >> One follow up question. May be a dumb one. What could be the potential >> problems with ALLOW_MEMORY_GROWTH missing in PThreads mode? I see that when >> the Wasm is instantiated, the overall memory that the Chrome tab was taking >> was similar to the one taken by the WASM built with ALLOW_MEMORY_GROWTH. Is >> it that, we will not be able to instantiate WASM on low end devices if >> built with ALLOW_MEMORY_GROWTH=0? >> >> Greatly appreciate your help Alon! >> >> Thanks, >> Prashanth Nethi >> >> On Saturday, August 22, 2020 at 1:50:57 AM UTC+5:30 [email protected] >> wrote: >> >>> I think you can do any number up to 2GB, including 2GB - 64Kb. So the >>> limit isn't 1GB, unless you see that on some specific browser? Could be a >>> bug. >>> >>> It should soon be possible to do up to 4GB for the initial memory >>> (without growth), thanks to a spec change, >>> https://github.com/WebAssembly/spec/pull/1174 >>> >>> On Thu, Aug 20, 2020 at 8:10 AM Prashanth Nethi <[email protected]> >>> wrote: >>> >>>> Hi, >>>> >>>> I am currently building WASM with the following flags, to enable >>>> PThreads in Wasm. >>>> -s USING_PTHREADS=1 -s INITIAL_MEMORY=1999MB -s MAXIMUM_MEMORY=2GB. >>>> >>>> This works wonderfully for our use cases! In fact we are able to get 2x >>>> performance in some cases! >>>> >>>> When I checked the max memory that the Wasm could use, with PThreads >>>> enabled, it got capped at 1 GB. I am seeing that when the WASM is built >>>> with ALLOW_MEMORY_GROWTH, the Wasm can use upto 2GB. I know that >>>> ALLOW_MEMORY_GROWTH with USE_PTHREADS is discouraged so can't look at that >>>> as a possible solution. >>>> >>>> Is there anyway I can get Wasm to use 2GB (or even potentially 4GB in >>>> the future) with PThreads enabled? Is it that I am missing using some >>>> configuration options? >>>> >>>> I am really hoping there is a way to increase the WASM cap to 2GB, as >>>> using PThreads, solves our use cases in a big way. >>>> >>>> Thanks, >>>> Prashanth Nethi >>>> >>>> -- >>>> You received this message because you are subscribed to the Google >>>> Groups "emscripten-discuss" group. >>>> To unsubscribe from this group and stop receiving emails from it, send >>>> an email to [email protected]. >>>> To view this discussion on the web visit >>>> https://groups.google.com/d/msgid/emscripten-discuss/730a6796-5b14-4a9e-a1d8-298415c67cd1n%40googlegroups.com >>>> >>>> <https://groups.google.com/d/msgid/emscripten-discuss/730a6796-5b14-4a9e-a1d8-298415c67cd1n%40googlegroups.com?utm_medium=email&utm_source=footer> >>>> . >>>> >>> -- >> You received this message because you are subscribed to the Google Groups >> "emscripten-discuss" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> > To view this discussion on the web visit >> https://groups.google.com/d/msgid/emscripten-discuss/86a9fc74-2036-4749-8212-29f6802615d0n%40googlegroups.com >> >> <https://groups.google.com/d/msgid/emscripten-discuss/86a9fc74-2036-4749-8212-29f6802615d0n%40googlegroups.com?utm_medium=email&utm_source=footer> >> . >> > -- You received this message because you are subscribed to the Google Groups "emscripten-discuss" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/emscripten-discuss/9ddd4487-ff48-4b05-a138-900103ec2a4dn%40googlegroups.com.
