Indeed there was a bug causing the number of pthread_create to grow, thus allocating more threads in the pool didn't work. Thanks for clarifying this, now we've found the proper thread pool size that works.
On Wednesday, September 16, 2015 at 2:47:38 PM UTC-4, jj wrote: > > Hmm, are you running two separate .js files, one for the main page and > another for the physX library, and they are separate asm.js modules? That > would explain two separate pthread spawn pools. > > Another way to get multiple such messages is to first create a spawn pool > of X threads, but then pthread_create X+1 or more threads. As the pool runs > out, pthread_create will end up calling to the pool to add more. Perhaps > the code is over-allocating threads by one more than the current pool size > is? > > 2015-09-16 17:48 GMT+03:00 Robert Goulet <[email protected] > <javascript:>>: > >> Yes sorry if that wasn't clear... >> >> What I mean is that when I start the final executable .js file, it does >> print in the Nightly console the intended amount of threads. We set it to 4 >> so we get this: >> >> Preallocating 4 workers for a pthread spawn pool. >> >> However, a bit further, when the .js code loads the PhysX library (and I >> traced it down to the pthread_create inside PhysX), another message is >> printed: >> >> Preallocating 1 workers for a pthread spawn pool. >> >> I don't understand why this message would appear more than once, and >> specifically, why the second one says 1 instead of 4. Even if we put larger >> number, it always print this second message. It also takes a ridiculous >> amount of additional memory for each additional thread we add to this >> startup pool. That seems strange. >> >> On Wednesday, September 16, 2015 at 8:41:36 AM UTC-4, jj wrote: >>> >>> The -s PTHREAD_POOL_SIZE option is a linker option that can be used only >>> when linking to the final .js/.html file. If specified in other stages >>> (compiling, linking a static library), it is ignored. If you are specifying >>> -s PTHREAD_POOL_SIZE=4 to final .js link, and still are seeing a message >>> that it is pooling up only one thread at .js startup time, then it is a >>> clear bug. Although I am not sure I understand, since you first mention >>> that "we see that message that it is allocating 1 thread", but in the >>> later sentence "At least it does report allocating 4 threads when the >>> executable gets loaded initially"? >>> >>> 2015-09-15 16:12 GMT+03:00 Robert Goulet <[email protected]>: >>> >>>> We do properly set -s USE_PTHREAD=1 on all static lib/executable >>>> compile and link time arguments. True for PTHREAD_POOL_SIZE as well. >>>> But still, when PhysX creates a thread, we see that message that it is >>>> allocating 1 thread, even though we told it to allocate 4 threads. Maybe >>>> we >>>> need to disregard the message or its a bug. At least it does report >>>> allocating 4 threads when the executable gets loaded initially. >>>> >>>> As for using that command to empty the proxy calls, I will try that and >>>> see if it fixes the deadlock. Thanks for the help! >>>> >>>> On Monday, September 14, 2015 at 5:43:17 PM UTC-4, jj wrote: >>>>> >>>>> Iirc I had to do a few changes to fix up my PhysX build system to >>>>> correctly build atomic operations. When building static libraries with >>>>> threading enabled, make sure you have -s USE_PTHREADS=1 specified at both >>>>> compile and link time of the library to be as consistent as possible, as >>>>> well as link time of the final js/html output, since that is in the end >>>>> what dictates whether atomic ops or dummy no-ops are built. >>>>> >>>>> However looking at this code, I remember that this is due to hitting a >>>>> fairly exotic issue that is a result of bug >>>>> https://github.com/kripken/emscripten/issues/3495 . Reading the docs, >>>>> I notice that I've failed to properly document this scenario, so I wrote >>>>> down the note >>>>> https://github.com/kripken/emscripten/issues/3495#issuecomment-140211370 >>>>> and pushed >>>>> https://github.com/kripken/emscripten/commit/8e531340fb042e3f0d6e570f3bc5c74b6a471a9a >>>>> >>>>> to point this out. Such lock-free loops in main threads are quite rare so >>>>> this does not occur in many places, but it is still annoying since it's >>>>> currently undetectable, and more or less requires auditing all places >>>>> where >>>>> the main thread might perform lock-free operations, and dropping in a >>>>> call >>>>> to the assist function to give worker threads room "to breathe". >>>>> >>>>> 2015-09-14 23:24 GMT+03:00 Robert Goulet <[email protected]>: >>>>> >>>>>> Alon, Juj, any idea what the problem could be? >>>>>> >>>>>> >>>>>> On Friday, September 11, 2015 at 4:07:51 PM UTC-4, Robert Goulet >>>>>> wrote: >>>>>>> >>>>>>> Hi all, >>>>>>> >>>>>>> Trying to use the pthread implementation, I stumbled upon this >>>>>>> problem described here: >>>>>>> >>>>>>> *When -s PTHREAD_POOL_SIZE=<integer> is not specified and >>>>>>> pthread_create() is called, the new thread will not actually start to >>>>>>> run >>>>>>> immediately, but the main JS thread must yield execution back to >>>>>>> browser >>>>>>> first. This behavior is a result of #1049079 >>>>>>> <https://bugzilla.mozilla.org/show_bug.cgi?id=1049079 >>>>>>> <https://bugzilla.mozilla.org/show_bug.cgi?id=1049079>>.* >>>>>>> >>>>>>> It was easy to fix for our final executable using the >>>>>>> PTHREAD_POOL_SIZE option (the engine doesn't get stuck when we >>>>>>> start threads), however how is that supposed to work for a static >>>>>>> libraries? I ported the PhysX library, and when we initialize it with >>>>>>> our >>>>>>> engine, it get stuck in this loop : >>>>>>> >>>>>>> status = pthread_create(&getThread(this)->thread, &attr, >>>>>>> PxThreadStart, this); >>>>>>> PX_ASSERT(!status); >>>>>>> >>>>>>> // wait for thread to startup and write out TID >>>>>>> // otherwise TID dependent calls like setAffinity will fail. >>>>>>> while(atomicCompareExchange(&(getThread(this)->threadStarted), 1, 1) >>>>>>> == 0) // <-- stuck here >>>>>>> yield(); >>>>>>> >>>>>>> I specified the PTHREAD_POOL_SIZE when building PhysX libs, but it >>>>>>> doesn't seems to work. Is it supported, or a perhaps a bug? >>>>>>> >>>>>>> On a similar note....... When we start out final executable, we get >>>>>>> this message: >>>>>>> >>>>>>> Preallocating 4 workers for a pthread spawn pool. >>>>>>> >>>>>>> So far so good, but I also noticed this message in the console >>>>>>> exactly when PhysX creates a thread: >>>>>>> >>>>>>> Preallocating 1 workers for a pthread spawn pool. >>>>>>> >>>>>>> So it really looks like the static lib isn't using the pool we >>>>>>> assigned previously? Are we supposed to see this message more than once? >>>>>>> >>>>>>> Another point to note, if we use a starting thread pool size of 8, >>>>>>> Firefox Nightly stops saying out of memory, while 4 works. That seems >>>>>>> rather strange... >>>>>>> >>>>>>> Any help appreciated! >>>>>>> Thanks! >>>>>>> >>>>>>> -Robert Goulet >>>>>>> >>>>>> -- >>>>>> You received this message because you are subscribed to the Google >>>>>> Groups "emscripten-discuss" group. >>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>> send an email to [email protected]. >>>>>> For more options, visit https://groups.google.com/d/optout. >>>>>> >>>>> >>>>> -- >>>> You received this message because you are subscribed to the Google >>>> Groups "emscripten-discuss" group. >>>> To unsubscribe from this group and stop receiving emails from it, send >>>> an email to [email protected]. >>>> For more options, visit https://groups.google.com/d/optout. >>>> >>> >>> -- >> You received this message because you are subscribed to the Google Groups >> "emscripten-discuss" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected] <javascript:>. >> For more options, visit https://groups.google.com/d/optout. >> > > -- You received this message because you are subscribed to the Google Groups "emscripten-discuss" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
