Hi Alon, I used wasm-dis to disassemble my MAIN_MODULE, and issue is same as it was with my side_modules. Main_module built with compiler 1.39.7 has lot of extra imports and exports. I am using -s EXPORTED_FUNCTIONS to pass list of exported functions and also passing MAIN_MODULE = 2 flag at link time. Passing SIDE_MODULE=2 flag at linked time in my side modules solved the size problem and removed extra exports and imports. But it's not working with main module. What else could be the reasons for having these lot of export functions in main module. I am using compiler 1.39.7.
Regards, Rohit Saini On Thursday, March 26, 2020 at 11:47:00 PM UTC+5:30, Alon Zakai wrote: > > You can use wasm-dis (part of the emsdk download, under /upstream/bin or > /fastcomp/bin) to get the text form of the wasm file. Then the exports look > like > > (export ..) > > and they are all together, near the beginning of the file. > > > On Thu, Mar 26, 2020 at 9:56 AM Rohit Saini <[email protected] > <javascript:>> wrote: > >> Hi Alon, >> >> Sorry for waking up this thread after some time again. >> >> As I mentioned, by passing SIDE_MODULE=2, size of my side modules came >> back to same as with previous compiler. But in my main module I was already >> using MAIN_MODULE=2, but still my size increased a lot with new backend >> compiler. You did a symbol analysis on one of my side module and told me >> that it has lot of export functions. Can you tell me the way, how you did >> it, so that I can do this analysis on my main module at my end. >> >> Regards, >> Rohit Saini >> >> On Saturday, March 7, 2020 at 3:07:45 AM UTC+5:30, Alon Zakai wrote: >>> >>> >>> >>> On Fri, Mar 6, 2020 at 9:42 AM Rohit Saini <[email protected]> wrote: >>> >>>> Hi Alon, >>>> >>>> Thanks a lot. Passing SIDE_MODULE = 2 decreased the size of wasm a lot. >>>> But one thing I want to confirm, does something related to this changed >>>> with latest backend, because initially I was using SIDE_MODULE = 1, and >>>> size of wasm files was okk. >>>> >>> >>> That should not change with the backend, and I verified on a small >>> testcase now that fastcomp and upstream do the same thing. >>> >>> However, if you were using fastcomp on an older version (I tested >>> latest), then there were bugs that have been fixed. Perhaps you used an old >>> version where the dce happened anyhow, even though it should not have. I >>> seem to recall us fixing such a bug. >>> >>> >>>> Moreover I want to ask one more thing. As most of the things needs to >>>> be passed at compile time with new backend. Do I need to pass >>>> EXPORTED_FUNCTIONS=[_someFunction] >>>> at compile time too. >>>> >>>> >>> EXPORTED_FUNCTIONS only matters at link time. >>> >>> Regards, >>>> >>>> >>>> On Friday, March 6, 2020 at 4:33:03 AM UTC+5:30, Alon Zakai wrote: >>>>> >>>>> Thanks Rohit! >>>>> >>>>> One big difference here is the first has 130 exports while the second >>>>> has 1,113. >>>>> >>>>> For example, JP2KTileComponentSetEncodeOptions (C function) and >>>>> _ZN10IJP2KImage16CreateCodeStreamEv (C++ function) are present in the new >>>>> one, but not the old one. >>>>> >>>>> Are those in the list of exported functions? >>>>> >>>>> From the command above it looks like you're using SIDE_MODULE=1, and >>>>> not =2, to build these. If so then it won't DCE away things not in the >>>>> list >>>>> of exports you provide. I'm not sure offhand why that would be different >>>>> between the backends though. >>>>> >>>>> On Thu, Mar 5, 2020 at 4:04 AM Rohit Saini <[email protected]> >>>>> wrote: >>>>> >>>>>> Hi Alon, >>>>>> >>>>>> Here are the wasm files of one of my side module before >>>>>> (libjp2k.wasm) and after (libjp2k_new.wasm) the compiler update. Please >>>>>> tell if you need more info from my side. >>>>>> >>>>>> Regards, >>>>>> >>>>>> >>>>>> >>>>>> On Tuesday, March 3, 2020 at 3:15:39 AM UTC+5:30, Alon Zakai wrote: >>>>>>> >>>>>>> Rohit: Can you perhaps provide the wasm files before and after, >>>>>>> built with --profiling? (that flag ensures functions have names) >>>>>>> perhaps by >>>>>>> comparing them we can find what's gone wrong. >>>>>>> >>>>>>> On Sat, Feb 29, 2020 at 9:45 AM Rohit Saini <[email protected]> >>>>>>> wrote: >>>>>>> >>>>>>>> Hi Alon, >>>>>>>> >>>>>>>> I tried removing -llvm-lto 3 and used -Os while linking, but still >>>>>>>> there is no change in size. Trying --llvm-lto 1 didn't help either. >>>>>>>> >>>>>>>> Regards, >>>>>>>> Rohit Saini >>>>>>>> >>>>>>>> On Saturday, February 29, 2020 at 3:27:58 AM UTC+5:30, Alon Zakai >>>>>>>> wrote: >>>>>>>>> >>>>>>>>> Rohit: LTO might be the issue here as others mentioned. >>>>>>>>> Specifically, with the wasm backend, object files are wasm by >>>>>>>>> default. To >>>>>>>>> get LTO you must build the source files with -flto, so that they >>>>>>>>> contain >>>>>>>>> bitcode. (wasm-ld will link both wasm and bitcode files, doing LTO >>>>>>>>> where it >>>>>>>>> can, but not warning if there are no actual bitcode files to LTO >>>>>>>>> with.) >>>>>>>>> >>>>>>>>> Note that the LLVM versions are quite different, so LTO may behave >>>>>>>>> differently in the new backend. Might be worth trying --llvm-lto 1 >>>>>>>>> instead >>>>>>>>> of 3. >>>>>>>>> >>>>>>>>> Also, might be worth doing -Os (not -O3) during link. >>>>>>>>> >>>>>>>>> Gabriel: That does seem odd. Are you building source files with >>>>>>>>> -flto? Worth checking if the object files are bitcode, as maybe the >>>>>>>>> build >>>>>>>>> system somehow didn't pass the flag through? >>>>>>>>> >>>>>>>>> With that said, I haven't seen great results from LTO on the new >>>>>>>>> backend either. So yeah, maybe something is wrong? I opened >>>>>>>>> https://github.com/emscripten-core/emscripten/issues/10603 for >>>>>>>>> more discussion. >>>>>>>>> >>>>>>>>> >>>>>>>>> On Fri, Feb 28, 2020 at 3:57 AM Gabriel Cuvillier < >>>>>>>>> [email protected]> wrote: >>>>>>>>> >>>>>>>>>> Well, to me LTO *used to* be better. But for some reason, since >>>>>>>>>> the switch to upstream LLVM backend, it is no more very interesting. >>>>>>>>>> >>>>>>>>>> I have been surprised by this, and decided to disable LTO on all >>>>>>>>>> my projects since the switch. But maybe there's a bug lurking >>>>>>>>>> somewhere: >>>>>>>>>> LTO is simply not correctly done anymore (?) >>>>>>>>>> >>>>>>>>>> @Emscripten team: any thoughts on this possibility ? or is it >>>>>>>>>> simply because the Wasm backend now does things extremely well ? >>>>>>>>>> Le 28/02/2020 à 12:49, Floh a écrit : >>>>>>>>>> >>>>>>>>>> ...hm ok I did a quick check on my 8-bit emulators, and I have to >>>>>>>>>> agree. I'm seeing about 10% size agree for the generated WASM (which >>>>>>>>>> I'd be >>>>>>>>>> willing to eat if the performance would improve too), but >>>>>>>>>> performance isn't >>>>>>>>>> any noticeably better. I guess the WASM runtimes either do the >>>>>>>>>> inlining >>>>>>>>>> themselves, or can the function call overhead doesn't matter as much >>>>>>>>>> as it >>>>>>>>>> used to. >>>>>>>>>> >>>>>>>>>> On Friday, 28 February 2020 12:16:39 UTC+1, Floh wrote: >>>>>>>>>>> >>>>>>>>>>> When I last tested I generally saw quite noticeable improvements >>>>>>>>>>> for inlined code (which LTO does across compilation units). Most of >>>>>>>>>>> my code >>>>>>>>>>> is straight C without inline functions in headers. I guess this >>>>>>>>>>> scenario >>>>>>>>>>> benefits more from LTO than typical C++ code where usually a lot of >>>>>>>>>>> implementation code sits in inline/template functions (at the cost >>>>>>>>>>> of >>>>>>>>>>> increasing compile time for each compilation unit, versus >>>>>>>>>>> increasing >>>>>>>>>>> link-time once for LTO). >>>>>>>>>>> >>>>>>>>>>> I think it depends extremely on the code base at hand whether >>>>>>>>>>> LTO is useful or not :) >>>>>>>>>>> >>>>>>>>>>> On Friday, 28 February 2020 11:32:09 UTC+1, Gabriel CV wrote: >>>>>>>>>>>> >>>>>>>>>>>> Glad to! >>>>>>>>>>>> >>>>>>>>>>>> By the way, I don't find that LTO brings any performance >>>>>>>>>>>> improvements compared to a non-LTO builds (on latests Emscripten >>>>>>>>>>>> backend). >>>>>>>>>>>> Maybe +1% at most (more probably benchmark noise to me), but >>>>>>>>>>>> sometime more >>>>>>>>>>>> than 10% worse, and at the expense of a *significant *link >>>>>>>>>>>> time increase and *significant *binary size increase! >>>>>>>>>>>> >>>>>>>>>>>> So I am not sure this is really an interesting option anymore... >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Le 28/02/2020 à 11:14, Floh a écrit : >>>>>>>>>>>> >>>>>>>>>>>> Whoops, thanks for making me aware of WASM_OBJECT_FILES being 1 >>>>>>>>>>>> by default... I've been building with LTO all the time (since the >>>>>>>>>>>> asm.js >>>>>>>>>>>> days) and was expecting that the WASM backend would behave >>>>>>>>>>>> identically >>>>>>>>>>>> there. >>>>>>>>>>>> >>>>>>>>>>>> -Floh. >>>>>>>>>>>> >>>>>>>>>>>> On Friday, 28 February 2020 10:07:11 UTC+1, Gabriel CV wrote: >>>>>>>>>>>>> >>>>>>>>>>>>> Hi, >>>>>>>>>>>>> >>>>>>>>>>>>> Try to remove --llvm-lto 3, as it might increase binary size >>>>>>>>>>>>> significantly when combined with -O2 or -O3 (and I am not sure it >>>>>>>>>>>>> is very >>>>>>>>>>>>> usefull without using -DWASM_OBJECT_FILES=0) >>>>>>>>>>>>> Le 28/02/2020 à 08:12, Rohit Saini a écrit : >>>>>>>>>>>>> >>>>>>>>>>>>> Hi All, >>>>>>>>>>>>> >>>>>>>>>>>>> Previously we were using 1.38.28 emscripten version. Recently >>>>>>>>>>>>> we updated to latest llvm backend 1.39.7. But with new backend >>>>>>>>>>>>> there is >>>>>>>>>>>>> drastic increase in wasm size. Size of my side modules become >>>>>>>>>>>>> almost double >>>>>>>>>>>>> and size of my main module also increased from 2.7mb to almost >>>>>>>>>>>>> 4mb. Firstly >>>>>>>>>>>>> we are compiling to object files then we are linking them to make >>>>>>>>>>>>> wasm. >>>>>>>>>>>>> Below are the flags I am passing while compiling and linking. Is >>>>>>>>>>>>> this size >>>>>>>>>>>>> increase intentional or do I have to change compiling or linking >>>>>>>>>>>>> flags >>>>>>>>>>>>> someway. >>>>>>>>>>>>> >>>>>>>>>>>>> Compiling flags for main module: >>>>>>>>>>>>> -Oz -fPIC >>>>>>>>>>>>> -s DISABLE_EXCEPTION_CATCHING=0 -Wno-builtin-macro-redefined >>>>>>>>>>>>> -Wno-dollar-in-identifier-extension >>>>>>>>>>>>> >>>>>>>>>>>>> Compiling flags for side module: >>>>>>>>>>>>> -std=c++14 -fPIC >>>>>>>>>>>>> >>>>>>>>>>>>> Linking flags for main module: >>>>>>>>>>>>> >>>>>>>>>>>>> -O3 -s EVAL_CTORS=0 --closure 0 -s ALIASING_FUNCTION_POINTERS=0 >>>>>>>>>>>>> -s ELIMINATE_DUPLICATE_FUNCTIONS=1 -s >>>>>>>>>>>>> ELIMINATE_DUPLICATE_FUNCTIONS_PASSES=12 --llvm-lto 3 -s >>>>>>>>>>>>> FORCE_FILESYSTEM=1 -s ERROR_ON_UNDEFINED_SYMBOLS=0 -s >>>>>>>>>>>>> 'EXTRA_EXPORTED_RUNTIME_METHODS=["getTempRet0","setTempRet0", >>>>>>>>>>>>> "cwrap"]' -s >>>>>>>>>>>>> [email protected] >>>>>>>>>>>>> -s DISABLE_EXCEPTION_CATCHING=2 -s >>>>>>>>>>>>> [email protected] >>>>>>>>>>>>> >>>>>>>>>>>>> Linking flags for side module: >>>>>>>>>>>>> -s SIDE_MODULE=1 -s WASM=1 -Oz -s EVAL_CTORS=0 --closure 0 >>>>>>>>>>>>> -s ALIASING_FUNCTION_POINTERS=0 -s >>>>>>>>>>>>> ELIMINATE_DUPLICATE_FUNCTIONS=1 -s >>>>>>>>>>>>> ELIMINATE_DUPLICATE_FUNCTIONS_PASSES=12 --llvm-lto 3 -s >>>>>>>>>>>>> ERROR_ON_UNDEFINED_SYMBOLS=0 >>>>>>>>>>>>> -- >>>>>>>>>>>>> You received this message because you are subscribed to the >>>>>>>>>>>>> Google Groups "emscripten-discuss" group. >>>>>>>>>>>>> To unsubscribe from this group and stop receiving emails from >>>>>>>>>>>>> it, send an email to >>>>>>>>>>>>> [email protected]. >>>>>>>>>>>>> To view this discussion on the web visit >>>>>>>>>>>>> https://groups.google.com/d/msgid/emscripten-discuss/5f30a113-7203-4eba-9426-9f626c51ec26%40googlegroups.com >>>>>>>>>>>>> >>>>>>>>>>>>> <https://groups.google.com/d/msgid/emscripten-discuss/5f30a113-7203-4eba-9426-9f626c51ec26%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>>>>>>>>>> . >>>>>>>>>>>>> >>>>>>>>>>>>> -- >>>>>>>>>>>> You received this message because you are subscribed to the >>>>>>>>>>>> Google Groups "emscripten-discuss" group. >>>>>>>>>>>> To unsubscribe from this group and stop receiving emails from >>>>>>>>>>>> it, send an email to >>>>>>>>>>>> [email protected]. >>>>>>>>>>>> To view this discussion on the web visit >>>>>>>>>>>> https://groups.google.com/d/msgid/emscripten-discuss/e8a9f44f-a8f5-46f2-b440-e2356f285d28%40googlegroups.com >>>>>>>>>>>> >>>>>>>>>>>> <https://groups.google.com/d/msgid/emscripten-discuss/e8a9f44f-a8f5-46f2-b440-e2356f285d28%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>>>>>>>>> . >>>>>>>>>>>> >>>>>>>>>>>> -- >>>>>>>>>> You received this message because you are subscribed to the >>>>>>>>>> Google Groups "emscripten-discuss" group. >>>>>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>>>>> send an email to [email protected]. >>>>>>>>>> To view this discussion on the web visit >>>>>>>>>> https://groups.google.com/d/msgid/emscripten-discuss/38cfb2aa-6b9c-4dfb-a978-1c9c520d3693%40googlegroups.com >>>>>>>>>> >>>>>>>>>> <https://groups.google.com/d/msgid/emscripten-discuss/38cfb2aa-6b9c-4dfb-a978-1c9c520d3693%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>>>>>>> . >>>>>>>>>> >>>>>>>>>> -- >>>>>>>>>> You received this message because you are subscribed to the >>>>>>>>>> Google Groups "emscripten-discuss" group. >>>>>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>>>>> send an email to [email protected]. >>>>>>>>>> To view this discussion on the web visit >>>>>>>>>> https://groups.google.com/d/msgid/emscripten-discuss/2e3a6f1d-869e-06f5-5341-a031aaa8fc09%40gmail.com >>>>>>>>>> >>>>>>>>>> <https://groups.google.com/d/msgid/emscripten-discuss/2e3a6f1d-869e-06f5-5341-a031aaa8fc09%40gmail.com?utm_medium=email&utm_source=footer> >>>>>>>>>> . >>>>>>>>>> >>>>>>>>> -- >>>>>>>> You received this message because you are subscribed to the Google >>>>>>>> Groups "emscripten-discuss" group. >>>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>>> send an email to [email protected]. >>>>>>>> To view this discussion on the web visit >>>>>>>> https://groups.google.com/d/msgid/emscripten-discuss/8d6187d7-019e-4191-9d10-ec5e655980f3%40googlegroups.com >>>>>>>> >>>>>>>> <https://groups.google.com/d/msgid/emscripten-discuss/8d6187d7-019e-4191-9d10-ec5e655980f3%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>>>>> . >>>>>>>> >>>>>>> -- >>>>>> You received this message because you are subscribed to the Google >>>>>> Groups "emscripten-discuss" group. >>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>> send an email to [email protected]. >>>>>> To view this discussion on the web visit >>>>>> https://groups.google.com/d/msgid/emscripten-discuss/144edb11-1b79-49ef-98e5-e220357eb713%40googlegroups.com >>>>>> >>>>>> <https://groups.google.com/d/msgid/emscripten-discuss/144edb11-1b79-49ef-98e5-e220357eb713%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>>> . >>>>>> >>>>> -- >>>> You received this message because you are subscribed to the Google >>>> Groups "emscripten-discuss" group. >>>> To unsubscribe from this group and stop receiving emails from it, send >>>> an email to [email protected]. >>>> To view this discussion on the web visit >>>> https://groups.google.com/d/msgid/emscripten-discuss/dfb54b90-be3a-4b97-b15b-c436fd7ad6b5%40googlegroups.com >>>> >>>> <https://groups.google.com/d/msgid/emscripten-discuss/dfb54b90-be3a-4b97-b15b-c436fd7ad6b5%40googlegroups.com?utm_medium=email&utm_source=footer> >>>> . >>>> >>> -- >> You received this message because you are subscribed to the Google Groups >> "emscripten-discuss" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected] <javascript:>. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/emscripten-discuss/880f1ce8-c8fe-44b8-bb9d-c040f24731e6%40googlegroups.com >> >> <https://groups.google.com/d/msgid/emscripten-discuss/880f1ce8-c8fe-44b8-bb9d-c040f24731e6%40googlegroups.com?utm_medium=email&utm_source=footer> >> . >> > -- You received this message because you are subscribed to the Google Groups "emscripten-discuss" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/emscripten-discuss/2a346383-0886-4af3-a043-d899ea46c183%40googlegroups.com.
