On Wed, Jan 22, 2020 at 2:00 PM Patrik Weiskircher <[email protected]> wrote:
> > [PassRunner] running pass: legalize-js-interface... 863.403 seconds. > > I wonder if that's an error in the measurement, otherwise, that's very > surprising, I don't remember any non-linear algorithms there... > > It is NOT an error in measurement, it really takes that long. I've been > profiling using Instruments on my Mac and from what I see most of the time > is spent here: > > > https://github.com/WebAssembly/binaryen/blob/e8f9d207427bda2f6e22c28ff0210b294b1f70e1/src/passes/LegalizeJSInterface.cpp#L121-L133 > > Specifically modifying the `unordered_map`. > > If I understand the code correctly, this basically means we have a *ton* > of functions that we're trying to export and therefor have to "legalize"? > Is it possible that we just have our project a little misconfigured and > instead of only exporting methods with Embind everything is exported? I > couldn't explain why the unordered_map would be used for 15 minutes(!) > otherwise. > > anyhow, if you share the binary I can investigate, should be easy to >> narrow something that noticeable down ;) >> > > Would you mind if I send it to you personally instead of to the mailing > list? If you think it is even necessary anymore seeing the above statement. > > Feel free to send it to me personally, yeah. I still need to figure out why that code is taking nonlinear time - that should just be doing a linear scan over the wasm module... Is this built with MAIN_MODULE perhaps? That would increase the # of exports. Still, no number can explain that slowdown, so it's still a mystery to me. > > On Wed, Jan 22, 2020 at 4:50 PM Alon Zakai <[email protected]> wrote: > >> > The code that I am compiling is proprietary but I feel like at that >> stage it's just one big object file without any source? >> >> Yeah, if you look at the input to wasm-emscripten-finalize, it's very >> close to the final wasm binary you'd ship on the web. So it's all compiled >> into wasm binary code at that point. >> >> One possible difference is this binary might contain function names, >> which might be stripped out by emcc later depending on the options. You >> might also want to take a look at the binary yourself to see if there's >> anything that looks like you can't share it, just to be sure. >> >> > [PassRunner] running pass: legalize-js-interface... 863.403 seconds. >> >> I wonder if that's an error in the measurement, otherwise, that's very >> surprising, I don't remember any non-linear algorithms there... anyhow, if >> you share the binary I can investigate, should be easy to narrow something >> that noticeable down ;) >> >> On Wed, Jan 22, 2020 at 1:23 PM Patrik Weiskircher <[email protected]> >> wrote: >> >>> >>> If you built binaryen by yourself, I'd make sure that's not a debug >>>> build. >>>> >>> >>> Simply did `emsdk install latest` and nothing is built by myself. >>> >>> Another thing is to check you don't set BINARYEN_CORES=1 in the >>>> environment (which would prevent an almost linear speedup with more cores). >>>> >>> >>> Don't think this is a problem here. I see `top` report 800% CPU use so >>> it's definitely using all Cores. >>> >>> And in a very big project perhaps it takes enough memory to cause >>>> swapping, so checking peak memory usage etc. might be interesting. >>>> >>> >>> It's only at 15.8% memory usage. >>> >>> If those aren't it, I'd run it in a profiler to see what's taking so >>>> long. If you can share the file I can take a look too. >>>> >>> >>> I honestly don't know enough about what format the file is in to make a >>> good decision here. The code that I am compiling is proprietary but I feel >>> like at that stage it's just one big object file without any source? As >>> long as we're not leaking source code I'm very happy to share the file :) >>> >>> I recently tracing channels to binaryen and some minimal tracing to >>>> `wasm-emscripten-finalize`. You might be able to get some clues from >>>> adding `--debug=emscripten` to the command line. >>>> >>> >>> I just added this! Here is the result: >>> >>> [PassRunner] running passes... >>>> [PassRunner] running pass: legalize-js-interface... 863.403 seconds. >>>> [PassRunner] (validating) >>>> [PassRunner] passes took 863.403 seconds. >>>> [PassRunner] (final validation) >>>> >>> >>> >>> On Wed, Jan 22, 2020 at 4:00 PM 'Sam Clegg' via emscripten-discuss < >>> [email protected]> wrote: >>> >>>> I recently tracing channels to binaryen and some minimal tracing to >>>> `wasm-emscripten-finalize`. You might be able to get some clues from >>>> adding `--debug=emscripten` to the command line. >>>> >>>> On Wed, Jan 22, 2020 at 12:52 PM Alon Zakai <[email protected]> >>>> wrote: >>>> >>>>> Oh, wow, it shouldn't be anything like that... >>>>> >>>>> If you built binaryen by yourself, I'd make sure that's not a debug >>>>> build. Another thing is to check you don't set BINARYEN_CORES=1 in the >>>>> environment (which would prevent an almost linear speedup with more >>>>> cores). >>>>> And in a very big project perhaps it takes enough memory to cause >>>>> swapping, >>>>> so checking peak memory usage etc. might be interesting. >>>>> >>>>> If those aren't it, I'd run it in a profiler to see what's taking so >>>>> long. If you can share the file I can take a look too. >>>>> >>>>> - Alon >>>>> >>>>> >>>>> >>>>> >>>>> On Wed, Jan 22, 2020 at 11:31 AM Patrik Weiskircher < >>>>> [email protected]> wrote: >>>>> >>>>>> Hi! >>>>>> >>>>>> I'm working at PSPDFKit trying to upgrade our Web viewer from >>>>>> Emscripten version 1.38.31 to 1.39.6. >>>>>> I got everything working so far, I'm just having a little trouble >>>>>> with the linking speed. Everything goes fine and quick >>>>>> enough until we hit linking the library - specifically the >>>>>> `wasm-emscripten-finalize` step. >>>>>> >>>>>> The exact command line is this: >>>>>> > /opt/emsdk/upstream/bin/wasm-emscripten-finalize --detect-features >>>>>> --global-base=1024 --check-stack-overflow \ >>>>>> > /tmp/emscripten_temp_rjgvGG/pspdfkit.wasm.wasm -o >>>>>> /tmp/emscripten_temp_rjgvGG/pspdfkit.wasm.wasm.o.wasm >>>>>> >>>>>> This takes around 15 minutes. While this isn't a deal breaker, it >>>>>> does make our CI quite slow and keeps agents occupied >>>>>> longer than necessary. It'll also not be great when we have to debug >>>>>> a problem on Emscripten in the future. >>>>>> >>>>>> I looked around on the GitHub issue board and also on the mailing >>>>>> list but I couldn't find any information about this. I >>>>>> understand we have quite a big code base and things will take some >>>>>> time - I'm just wondering if anyone has any tips >>>>>> on how I could debug what *exactly* takes this long or how to make >>>>>> this any quicker? >>>>>> >>>>>> Thanks so much! >>>>>> Patrik >>>>>> >>>>>> >>>>>> -- >>>>>> You received this message because you are subscribed to the Google >>>>>> Groups "emscripten-discuss" group. >>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>> send an email to [email protected]. >>>>>> To view this discussion on the web visit >>>>>> https://groups.google.com/d/msgid/emscripten-discuss/cca83aa0-c022-4a94-8cc6-42063b5ccebc%40googlegroups.com >>>>>> <https://groups.google.com/d/msgid/emscripten-discuss/cca83aa0-c022-4a94-8cc6-42063b5ccebc%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>>> . >>>>>> >>>>> -- >>>>> You received this message because you are subscribed to the Google >>>>> Groups "emscripten-discuss" group. >>>>> To unsubscribe from this group and stop receiving emails from it, send >>>>> an email to [email protected]. >>>>> To view this discussion on the web visit >>>>> https://groups.google.com/d/msgid/emscripten-discuss/CAEX4NpREWRPRu1M%3DEx2t%2BQXkt7pQ64sgFC%2BaGU6h_H5vbX9ocA%40mail.gmail.com >>>>> <https://groups.google.com/d/msgid/emscripten-discuss/CAEX4NpREWRPRu1M%3DEx2t%2BQXkt7pQ64sgFC%2BaGU6h_H5vbX9ocA%40mail.gmail.com?utm_medium=email&utm_source=footer> >>>>> . >>>>> >>>> -- >>>> You received this message because you are subscribed to a topic in the >>>> Google Groups "emscripten-discuss" group. >>>> To unsubscribe from this topic, visit >>>> https://groups.google.com/d/topic/emscripten-discuss/Tj7C5Ij4sx8/unsubscribe >>>> . >>>> To unsubscribe from this group and all its topics, send an email to >>>> [email protected]. >>>> To view this discussion on the web visit >>>> https://groups.google.com/d/msgid/emscripten-discuss/CAL_va28KeT9VfF45%3D%3DAEmVSML79uB9L%2B%2BWJV6EG6%3DL3NoGtvjg%40mail.gmail.com >>>> <https://groups.google.com/d/msgid/emscripten-discuss/CAL_va28KeT9VfF45%3D%3DAEmVSML79uB9L%2B%2BWJV6EG6%3DL3NoGtvjg%40mail.gmail.com?utm_medium=email&utm_source=footer> >>>> . >>>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "emscripten-discuss" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to [email protected]. >>> To view this discussion on the web visit >>> https://groups.google.com/d/msgid/emscripten-discuss/CANMzfzh6oQ6656xYf-C7T7eo5Es57TsnkkrNzFiJx64%3Dcx4whA%40mail.gmail.com >>> <https://groups.google.com/d/msgid/emscripten-discuss/CANMzfzh6oQ6656xYf-C7T7eo5Es57TsnkkrNzFiJx64%3Dcx4whA%40mail.gmail.com?utm_medium=email&utm_source=footer> >>> . >>> >> -- >> You received this message because you are subscribed to a topic in the >> Google Groups "emscripten-discuss" group. >> To unsubscribe from this topic, visit >> https://groups.google.com/d/topic/emscripten-discuss/Tj7C5Ij4sx8/unsubscribe >> . >> To unsubscribe from this group and all its topics, send an email to >> [email protected]. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/emscripten-discuss/CAEX4NpTqGBC4eJGJE4hrCCY_k4pA%3D4nXtBxGemSYqKQw_juGoQ%40mail.gmail.com >> <https://groups.google.com/d/msgid/emscripten-discuss/CAEX4NpTqGBC4eJGJE4hrCCY_k4pA%3D4nXtBxGemSYqKQw_juGoQ%40mail.gmail.com?utm_medium=email&utm_source=footer> >> . >> > -- > You received this message because you are subscribed to the Google Groups > "emscripten-discuss" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/emscripten-discuss/CANMzfzieX98ksvWvCQcD9StwOA95bjy7mkZ5kUvweZyvok71XQ%40mail.gmail.com > <https://groups.google.com/d/msgid/emscripten-discuss/CANMzfzieX98ksvWvCQcD9StwOA95bjy7mkZ5kUvweZyvok71XQ%40mail.gmail.com?utm_medium=email&utm_source=footer> > . > -- You received this message because you are subscribed to the Google Groups "emscripten-discuss" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/emscripten-discuss/CAEX4NpTE3GfoebGiKutkQEhT5cPkFAK4ohvJLHK-HdkffnZCwA%40mail.gmail.com.
