Just sent the file. Thank you for looking into this, Alon! > On Jan 22, 2020, at 5:56 PM, Alon Zakai <[email protected]> wrote: > > > > >> On Wed, Jan 22, 2020 at 2:00 PM Patrik Weiskircher <[email protected]> >> wrote: >> > [PassRunner] running pass: legalize-js-interface... 863.403 seconds. >> > I wonder if that's an error in the measurement, otherwise, that's very >> > surprising, I don't remember any non-linear algorithms there... >> >> It is NOT an error in measurement, it really takes that long. I've been >> profiling using Instruments on my Mac and from what I see most of the time >> is spent here: >> >> https://github.com/WebAssembly/binaryen/blob/e8f9d207427bda2f6e22c28ff0210b294b1f70e1/src/passes/LegalizeJSInterface.cpp#L121-L133 >> >> Specifically modifying the `unordered_map`. >> >> If I understand the code correctly, this basically means we have a *ton* of >> functions that we're trying to export and therefor have to "legalize"? Is it >> possible that we just have our project a little misconfigured and instead of >> only exporting methods with Embind everything is exported? I couldn't >> explain why the unordered_map would be used for 15 minutes(!) otherwise. >> >>> anyhow, if you share the binary I can investigate, should be easy to narrow >>> something that noticeable down ;) >> >> Would you mind if I send it to you personally instead of to the mailing >> list? If you think it is even necessary anymore seeing the above statement. >> > > Feel free to send it to me personally, yeah. I still need to figure out why > that code is taking nonlinear time - that should just be doing a linear scan > over the wasm module... > > Is this built with MAIN_MODULE perhaps? That would increase the # of exports. > Still, no number can explain that slowdown, so it's still a mystery to me. > >> >> >>> On Wed, Jan 22, 2020 at 4:50 PM Alon Zakai <[email protected]> wrote: >>> > The code that I am compiling is proprietary but I feel like at that stage >>> > it's just one big object file without any source? >>> >>> Yeah, if you look at the input to wasm-emscripten-finalize, it's very close >>> to the final wasm binary you'd ship on the web. So it's all compiled into >>> wasm binary code at that point. >>> >>> One possible difference is this binary might contain function names, which >>> might be stripped out by emcc later depending on the options. You might >>> also want to take a look at the binary yourself to see if there's anything >>> that looks like you can't share it, just to be sure. >>> >>> > [PassRunner] running pass: legalize-js-interface... 863.403 seconds. >>> >>> I wonder if that's an error in the measurement, otherwise, that's very >>> surprising, I don't remember any non-linear algorithms there... anyhow, if >>> you share the binary I can investigate, should be easy to narrow something >>> that noticeable down ;) >>> >>>> On Wed, Jan 22, 2020 at 1:23 PM Patrik Weiskircher <[email protected]> >>>> wrote: >>>> >>>>> If you built binaryen by yourself, I'd make sure that's not a debug >>>>> build. >>>> >>>> Simply did `emsdk install latest` and nothing is built by myself. >>>> >>>>> Another thing is to check you don't set BINARYEN_CORES=1 in the >>>>> environment (which would prevent an almost linear speedup with more >>>>> cores). >>>> >>>> Don't think this is a problem here. I see `top` report 800% CPU use so >>>> it's definitely using all Cores. >>>> >>>>> And in a very big project perhaps it takes enough memory to cause >>>>> swapping, so checking peak memory usage etc. might be interesting. >>>> >>>> It's only at 15.8% memory usage. >>>> >>>>> If those aren't it, I'd run it in a profiler to see what's taking so >>>>> long. If you can share the file I can take a look too. >>>> >>>> I honestly don't know enough about what format the file is in to make a >>>> good decision here. The code that I am compiling is proprietary but I feel >>>> like at that stage it's just one big object file without any source? As >>>> long as we're not leaking source code I'm very happy to share the file :) >>>> >>>>> I recently tracing channels to binaryen and some minimal tracing to >>>>> `wasm-emscripten-finalize`. You might be able to get some clues from >>>>> adding `--debug=emscripten` to the command line. >>>> >>>> I just added this! Here is the result: >>>> >>>>> [PassRunner] running passes... >>>>> [PassRunner] running pass: legalize-js-interface... 863.403 seconds. >>>>> [PassRunner] (validating) >>>>> [PassRunner] passes took 863.403 seconds. >>>>> [PassRunner] (final validation) >>>> >>>> >>>>> On Wed, Jan 22, 2020 at 4:00 PM 'Sam Clegg' via emscripten-discuss >>>>> <[email protected]> wrote: >>>>> I recently tracing channels to binaryen and some minimal tracing to >>>>> `wasm-emscripten-finalize`. You might be able to get some clues from >>>>> adding `--debug=emscripten` to the command line. >>>>> >>>>>> On Wed, Jan 22, 2020 at 12:52 PM Alon Zakai <[email protected]> wrote: >>>>>> Oh, wow, it shouldn't be anything like that... >>>>>> >>>>>> If you built binaryen by yourself, I'd make sure that's not a debug >>>>>> build. Another thing is to check you don't set BINARYEN_CORES=1 in the >>>>>> environment (which would prevent an almost linear speedup with more >>>>>> cores). And in a very big project perhaps it takes enough memory to >>>>>> cause swapping, so checking peak memory usage etc. might be interesting. >>>>>> >>>>>> If those aren't it, I'd run it in a profiler to see what's taking so >>>>>> long. If you can share the file I can take a look too. >>>>>> >>>>>> - Alon >>>>>> >>>>>> >>>>>> >>>>>> >>>>>>> On Wed, Jan 22, 2020 at 11:31 AM Patrik Weiskircher >>>>>>> <[email protected]> wrote: >>>>>>> Hi! >>>>>>> >>>>>>> I'm working at PSPDFKit trying to upgrade our Web viewer from >>>>>>> Emscripten version 1.38.31 to 1.39.6. >>>>>>> I got everything working so far, I'm just having a little trouble with >>>>>>> the linking speed. Everything goes fine and quick >>>>>>> enough until we hit linking the library - specifically the >>>>>>> `wasm-emscripten-finalize` step. >>>>>>> >>>>>>> The exact command line is this: >>>>>>> > /opt/emsdk/upstream/bin/wasm-emscripten-finalize --detect-features >>>>>>> > --global-base=1024 --check-stack-overflow \ >>>>>>> > /tmp/emscripten_temp_rjgvGG/pspdfkit.wasm.wasm -o >>>>>>> > /tmp/emscripten_temp_rjgvGG/pspdfkit.wasm.wasm.o.wasm >>>>>>> >>>>>>> This takes around 15 minutes. While this isn't a deal breaker, it does >>>>>>> make our CI quite slow and keeps agents occupied >>>>>>> longer than necessary. It'll also not be great when we have to debug a >>>>>>> problem on Emscripten in the future. >>>>>>> >>>>>>> I looked around on the GitHub issue board and also on the mailing list >>>>>>> but I couldn't find any information about this. I >>>>>>> understand we have quite a big code base and things will take some time >>>>>>> - I'm just wondering if anyone has any tips >>>>>>> on how I could debug what *exactly* takes this long or how to make this >>>>>>> any quicker? >>>>>>> >>>>>>> Thanks so much! >>>>>>> Patrik >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> You received this message because you are subscribed to the Google >>>>>>> Groups "emscripten-discuss" group. >>>>>>> To unsubscribe from this group and stop receiving emails from it, send >>>>>>> an email to [email protected]. >>>>>>> To view this discussion on the web visit >>>>>>> https://groups.google.com/d/msgid/emscripten-discuss/cca83aa0-c022-4a94-8cc6-42063b5ccebc%40googlegroups.com. >>>>>> >>>>>> -- >>>>>> You received this message because you are subscribed to the Google >>>>>> Groups "emscripten-discuss" group. >>>>>> To unsubscribe from this group and stop receiving emails from it, send >>>>>> an email to [email protected]. >>>>>> To view this discussion on the web visit >>>>>> https://groups.google.com/d/msgid/emscripten-discuss/CAEX4NpREWRPRu1M%3DEx2t%2BQXkt7pQ64sgFC%2BaGU6h_H5vbX9ocA%40mail.gmail.com. >>>>> >>>>> -- >>>>> You received this message because you are subscribed to a topic in the >>>>> Google Groups "emscripten-discuss" group. >>>>> To unsubscribe from this topic, visit >>>>> https://groups.google.com/d/topic/emscripten-discuss/Tj7C5Ij4sx8/unsubscribe. >>>>> To unsubscribe from this group and all its topics, send an email to >>>>> [email protected]. >>>>> To view this discussion on the web visit >>>>> https://groups.google.com/d/msgid/emscripten-discuss/CAL_va28KeT9VfF45%3D%3DAEmVSML79uB9L%2B%2BWJV6EG6%3DL3NoGtvjg%40mail.gmail.com. >>>> >>>> -- >>>> You received this message because you are subscribed to the Google Groups >>>> "emscripten-discuss" group. >>>> To unsubscribe from this group and stop receiving emails from it, send an >>>> email to [email protected]. >>>> To view this discussion on the web visit >>>> https://groups.google.com/d/msgid/emscripten-discuss/CANMzfzh6oQ6656xYf-C7T7eo5Es57TsnkkrNzFiJx64%3Dcx4whA%40mail.gmail.com. >>> >>> -- >>> You received this message because you are subscribed to a topic in the >>> Google Groups "emscripten-discuss" group. >>> To unsubscribe from this topic, visit >>> https://groups.google.com/d/topic/emscripten-discuss/Tj7C5Ij4sx8/unsubscribe. >>> To unsubscribe from this group and all its topics, send an email to >>> [email protected]. >>> To view this discussion on the web visit >>> https://groups.google.com/d/msgid/emscripten-discuss/CAEX4NpTqGBC4eJGJE4hrCCY_k4pA%3D4nXtBxGemSYqKQw_juGoQ%40mail.gmail.com. >> >> -- >> You received this message because you are subscribed to the Google Groups >> "emscripten-discuss" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/emscripten-discuss/CANMzfzieX98ksvWvCQcD9StwOA95bjy7mkZ5kUvweZyvok71XQ%40mail.gmail.com. > > -- > You received this message because you are subscribed to a topic in the Google > Groups "emscripten-discuss" group. > To unsubscribe from this topic, visit > https://groups.google.com/d/topic/emscripten-discuss/Tj7C5Ij4sx8/unsubscribe. > To unsubscribe from this group and all its topics, send an email to > [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/emscripten-discuss/CAEX4NpTE3GfoebGiKutkQEhT5cPkFAK4ohvJLHK-HdkffnZCwA%40mail.gmail.com.
-- You received this message because you are subscribed to the Google Groups "emscripten-discuss" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/emscripten-discuss/8109E6E4-DBDE-420E-B3A8-276FD3DE8A2A%40pspdfkit.com.
