My guess is that's because of the behavior of std::vector and how it resizes. Over those appends it will malloc and free repeatedly and that may cause fragmentation that prevents a final larger size, which must be a single contiguous region. The second version allocates many smaller ones, not a single contiguous region.
- Alon On Tue, Aug 25, 2020 at 11:24 PM Prashanth Nethi <[email protected]> wrote: > Thanks Alon! So here is something very weird. I could get the memory usage > go all the way to 2GB when I changed my testing code. This was my original > test code. So basically I was just adding elements to std::vector > infinitely. > > class TestClass{ > private: > int t = 0; > }; > > struct Data { > int t; > TestClass obj; > }; > > typedef std::vector<Data> Vec; > > Vec someVec; > > using namespace std; > > int main() { > printf("hello, world!\n"); > > while(1){ > Data data; > someVec.push_back(data); > } > > return 0; > } > > With this code, the WASM memory was going all the way to 1GB. > > But when I changed the code to this, where I am writing some value after > acquiring memory, then I am able to see the memory usage go all the way up > to 2 GB. Could this be a bug? I am on emscripten 2.0. > > > int main() { > printf("hello, world!\n"); > char *p = nullptr; > int byteSize = 50 * 1024 * 1024; > while(1){ > p = new char(byteSize); > p[byteSize] = 20; > } > return 0; > } > > Also It is very encouraging to see that 4GB is considered for PThreads as > well! Thanks. > > One follow up question. May be a dumb one. What could be the potential > problems with ALLOW_MEMORY_GROWTH missing in PThreads mode? I see that when > the Wasm is instantiated, the overall memory that the Chrome tab was taking > was similar to the one taken by the WASM built with ALLOW_MEMORY_GROWTH. Is > it that, we will not be able to instantiate WASM on low end devices if > built with ALLOW_MEMORY_GROWTH=0? > > Greatly appreciate your help Alon! > > Thanks, > Prashanth Nethi > > On Saturday, August 22, 2020 at 1:50:57 AM UTC+5:30 [email protected] > wrote: > >> I think you can do any number up to 2GB, including 2GB - 64Kb. So the >> limit isn't 1GB, unless you see that on some specific browser? Could be a >> bug. >> >> It should soon be possible to do up to 4GB for the initial memory >> (without growth), thanks to a spec change, >> https://github.com/WebAssembly/spec/pull/1174 >> >> On Thu, Aug 20, 2020 at 8:10 AM Prashanth Nethi <[email protected]> >> wrote: >> >>> Hi, >>> >>> I am currently building WASM with the following flags, to enable >>> PThreads in Wasm. >>> -s USING_PTHREADS=1 -s INITIAL_MEMORY=1999MB -s MAXIMUM_MEMORY=2GB. >>> >>> This works wonderfully for our use cases! In fact we are able to get 2x >>> performance in some cases! >>> >>> When I checked the max memory that the Wasm could use, with PThreads >>> enabled, it got capped at 1 GB. I am seeing that when the WASM is built >>> with ALLOW_MEMORY_GROWTH, the Wasm can use upto 2GB. I know that >>> ALLOW_MEMORY_GROWTH with USE_PTHREADS is discouraged so can't look at that >>> as a possible solution. >>> >>> Is there anyway I can get Wasm to use 2GB (or even potentially 4GB in >>> the future) with PThreads enabled? Is it that I am missing using some >>> configuration options? >>> >>> I am really hoping there is a way to increase the WASM cap to 2GB, as >>> using PThreads, solves our use cases in a big way. >>> >>> Thanks, >>> Prashanth Nethi >>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "emscripten-discuss" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to [email protected]. >>> To view this discussion on the web visit >>> https://groups.google.com/d/msgid/emscripten-discuss/730a6796-5b14-4a9e-a1d8-298415c67cd1n%40googlegroups.com >>> <https://groups.google.com/d/msgid/emscripten-discuss/730a6796-5b14-4a9e-a1d8-298415c67cd1n%40googlegroups.com?utm_medium=email&utm_source=footer> >>> . >>> >> -- > You received this message because you are subscribed to the Google Groups > "emscripten-discuss" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/emscripten-discuss/86a9fc74-2036-4749-8212-29f6802615d0n%40googlegroups.com > <https://groups.google.com/d/msgid/emscripten-discuss/86a9fc74-2036-4749-8212-29f6802615d0n%40googlegroups.com?utm_medium=email&utm_source=footer> > . > -- You received this message because you are subscribed to the Google Groups "emscripten-discuss" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/emscripten-discuss/CAEX4NpSZx34yEBhiFv2P2syhoNKqnZo713ROhFMH8qa6G_8gHQ%40mail.gmail.com.
