> [PassRunner] running pass: legalize-js-interface... 863.403 seconds. > I wonder if that's an error in the measurement, otherwise, that's very surprising, I don't remember any non-linear algorithms there...
It is NOT an error in measurement, it really takes that long. I've been profiling using Instruments on my Mac and from what I see most of the time is spent here: https://github.com/WebAssembly/binaryen/blob/e8f9d207427bda2f6e22c28ff0210b294b1f70e1/src/passes/LegalizeJSInterface.cpp#L121-L133 Specifically modifying the `unordered_map`. If I understand the code correctly, this basically means we have a *ton* of functions that we're trying to export and therefor have to "legalize"? Is it possible that we just have our project a little misconfigured and instead of only exporting methods with Embind everything is exported? I couldn't explain why the unordered_map would be used for 15 minutes(!) otherwise. anyhow, if you share the binary I can investigate, should be easy to narrow > something that noticeable down ;) > Would you mind if I send it to you personally instead of to the mailing list? If you think it is even necessary anymore seeing the above statement. On Wed, Jan 22, 2020 at 4:50 PM Alon Zakai <[email protected]> wrote: > > The code that I am compiling is proprietary but I feel like at that > stage it's just one big object file without any source? > > Yeah, if you look at the input to wasm-emscripten-finalize, it's very > close to the final wasm binary you'd ship on the web. So it's all compiled > into wasm binary code at that point. > > One possible difference is this binary might contain function names, which > might be stripped out by emcc later depending on the options. You might > also want to take a look at the binary yourself to see if there's anything > that looks like you can't share it, just to be sure. > > > [PassRunner] running pass: legalize-js-interface... 863.403 seconds. > > I wonder if that's an error in the measurement, otherwise, that's very > surprising, I don't remember any non-linear algorithms there... anyhow, if > you share the binary I can investigate, should be easy to narrow something > that noticeable down ;) > > On Wed, Jan 22, 2020 at 1:23 PM Patrik Weiskircher <[email protected]> > wrote: > >> >> If you built binaryen by yourself, I'd make sure that's not a debug >>> build. >>> >> >> Simply did `emsdk install latest` and nothing is built by myself. >> >> Another thing is to check you don't set BINARYEN_CORES=1 in the >>> environment (which would prevent an almost linear speedup with more cores). >>> >> >> Don't think this is a problem here. I see `top` report 800% CPU use so >> it's definitely using all Cores. >> >> And in a very big project perhaps it takes enough memory to cause >>> swapping, so checking peak memory usage etc. might be interesting. >>> >> >> It's only at 15.8% memory usage. >> >> If those aren't it, I'd run it in a profiler to see what's taking so >>> long. If you can share the file I can take a look too. >>> >> >> I honestly don't know enough about what format the file is in to make a >> good decision here. The code that I am compiling is proprietary but I feel >> like at that stage it's just one big object file without any source? As >> long as we're not leaking source code I'm very happy to share the file :) >> >> I recently tracing channels to binaryen and some minimal tracing to >>> `wasm-emscripten-finalize`. You might be able to get some clues from >>> adding `--debug=emscripten` to the command line. >>> >> >> I just added this! Here is the result: >> >> [PassRunner] running passes... >>> [PassRunner] running pass: legalize-js-interface... 863.403 seconds. >>> [PassRunner] (validating) >>> [PassRunner] passes took 863.403 seconds. >>> [PassRunner] (final validation) >>> >> >> >> On Wed, Jan 22, 2020 at 4:00 PM 'Sam Clegg' via emscripten-discuss < >> [email protected]> wrote: >> >>> I recently tracing channels to binaryen and some minimal tracing to >>> `wasm-emscripten-finalize`. You might be able to get some clues from >>> adding `--debug=emscripten` to the command line. >>> >>> On Wed, Jan 22, 2020 at 12:52 PM Alon Zakai <[email protected]> wrote: >>> >>>> Oh, wow, it shouldn't be anything like that... >>>> >>>> If you built binaryen by yourself, I'd make sure that's not a debug >>>> build. Another thing is to check you don't set BINARYEN_CORES=1 in the >>>> environment (which would prevent an almost linear speedup with more cores). >>>> And in a very big project perhaps it takes enough memory to cause swapping, >>>> so checking peak memory usage etc. might be interesting. >>>> >>>> If those aren't it, I'd run it in a profiler to see what's taking so >>>> long. If you can share the file I can take a look too. >>>> >>>> - Alon >>>> >>>> >>>> >>>> >>>> On Wed, Jan 22, 2020 at 11:31 AM Patrik Weiskircher < >>>> [email protected]> wrote: >>>> >>>>> Hi! >>>>> >>>>> I'm working at PSPDFKit trying to upgrade our Web viewer from >>>>> Emscripten version 1.38.31 to 1.39.6. >>>>> I got everything working so far, I'm just having a little trouble with >>>>> the linking speed. Everything goes fine and quick >>>>> enough until we hit linking the library - specifically the >>>>> `wasm-emscripten-finalize` step. >>>>> >>>>> The exact command line is this: >>>>> > /opt/emsdk/upstream/bin/wasm-emscripten-finalize --detect-features >>>>> --global-base=1024 --check-stack-overflow \ >>>>> > /tmp/emscripten_temp_rjgvGG/pspdfkit.wasm.wasm -o >>>>> /tmp/emscripten_temp_rjgvGG/pspdfkit.wasm.wasm.o.wasm >>>>> >>>>> This takes around 15 minutes. While this isn't a deal breaker, it does >>>>> make our CI quite slow and keeps agents occupied >>>>> longer than necessary. It'll also not be great when we have to debug a >>>>> problem on Emscripten in the future. >>>>> >>>>> I looked around on the GitHub issue board and also on the mailing list >>>>> but I couldn't find any information about this. I >>>>> understand we have quite a big code base and things will take some >>>>> time - I'm just wondering if anyone has any tips >>>>> on how I could debug what *exactly* takes this long or how to make >>>>> this any quicker? >>>>> >>>>> Thanks so much! >>>>> Patrik >>>>> >>>>> >>>>> -- >>>>> You received this message because you are subscribed to the Google >>>>> Groups "emscripten-discuss" group. >>>>> To unsubscribe from this group and stop receiving emails from it, send >>>>> an email to [email protected]. >>>>> To view this discussion on the web visit >>>>> https://groups.google.com/d/msgid/emscripten-discuss/cca83aa0-c022-4a94-8cc6-42063b5ccebc%40googlegroups.com >>>>> <https://groups.google.com/d/msgid/emscripten-discuss/cca83aa0-c022-4a94-8cc6-42063b5ccebc%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>> . >>>>> >>>> -- >>>> You received this message because you are subscribed to the Google >>>> Groups "emscripten-discuss" group. >>>> To unsubscribe from this group and stop receiving emails from it, send >>>> an email to [email protected]. >>>> To view this discussion on the web visit >>>> https://groups.google.com/d/msgid/emscripten-discuss/CAEX4NpREWRPRu1M%3DEx2t%2BQXkt7pQ64sgFC%2BaGU6h_H5vbX9ocA%40mail.gmail.com >>>> <https://groups.google.com/d/msgid/emscripten-discuss/CAEX4NpREWRPRu1M%3DEx2t%2BQXkt7pQ64sgFC%2BaGU6h_H5vbX9ocA%40mail.gmail.com?utm_medium=email&utm_source=footer> >>>> . >>>> >>> -- >>> You received this message because you are subscribed to a topic in the >>> Google Groups "emscripten-discuss" group. >>> To unsubscribe from this topic, visit >>> https://groups.google.com/d/topic/emscripten-discuss/Tj7C5Ij4sx8/unsubscribe >>> . >>> To unsubscribe from this group and all its topics, send an email to >>> [email protected]. >>> To view this discussion on the web visit >>> https://groups.google.com/d/msgid/emscripten-discuss/CAL_va28KeT9VfF45%3D%3DAEmVSML79uB9L%2B%2BWJV6EG6%3DL3NoGtvjg%40mail.gmail.com >>> <https://groups.google.com/d/msgid/emscripten-discuss/CAL_va28KeT9VfF45%3D%3DAEmVSML79uB9L%2B%2BWJV6EG6%3DL3NoGtvjg%40mail.gmail.com?utm_medium=email&utm_source=footer> >>> . >>> >> -- >> You received this message because you are subscribed to the Google Groups >> "emscripten-discuss" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/emscripten-discuss/CANMzfzh6oQ6656xYf-C7T7eo5Es57TsnkkrNzFiJx64%3Dcx4whA%40mail.gmail.com >> <https://groups.google.com/d/msgid/emscripten-discuss/CANMzfzh6oQ6656xYf-C7T7eo5Es57TsnkkrNzFiJx64%3Dcx4whA%40mail.gmail.com?utm_medium=email&utm_source=footer> >> . >> > -- > You received this message because you are subscribed to a topic in the > Google Groups "emscripten-discuss" group. > To unsubscribe from this topic, visit > https://groups.google.com/d/topic/emscripten-discuss/Tj7C5Ij4sx8/unsubscribe > . > To unsubscribe from this group and all its topics, send an email to > [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/emscripten-discuss/CAEX4NpTqGBC4eJGJE4hrCCY_k4pA%3D4nXtBxGemSYqKQw_juGoQ%40mail.gmail.com > <https://groups.google.com/d/msgid/emscripten-discuss/CAEX4NpTqGBC4eJGJE4hrCCY_k4pA%3D4nXtBxGemSYqKQw_juGoQ%40mail.gmail.com?utm_medium=email&utm_source=footer> > . > -- You received this message because you are subscribed to the Google Groups "emscripten-discuss" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/emscripten-discuss/CANMzfzieX98ksvWvCQcD9StwOA95bjy7mkZ5kUvweZyvok71XQ%40mail.gmail.com.
