emcc (in the incoming branch) now understands -Wl,--start-group/-Wl,--end-group (and the -Wl,-( / -Wl,-) variations). Give it a try and let me know if you have any problems with it.
Ryan On Tuesday, April 22, 2014 10:44:39 AM UTC-7, Ryan Sturgell wrote: > > Agreed. I have a pretty clean change to support --start/end-group, I'll > send you a pull request. > > Ryan > > > On Tue, Apr 22, 2014 at 10:17 AM, Alon Zakai <[email protected]> wrote: > >> Regardless though, it would be a lot of work for us to switch to a gold >> plugin, both development and maintenance, so I doubt we would do it, as the >> only missing feature I am aware of is the groups issue being discussed >> here. We should fix that directly I think. >> >> - Alon >> >> >> >> On Tue, Apr 22, 2014 at 10:16 AM, Alon Zakai <[email protected]> wrote: >> >>> Thanks for looking into this, I'll ask some pnacl people I know. >>> >>> - Alon >>> >>> >>> >>> On Mon, Apr 14, 2014 at 4:15 PM, Ryan Sturgell >>> <[email protected]>wrote: >>> >>>> Hmm, yes, that file seems to have disappeared from that location! Let's >>>> try again: >>>> >>>> >>>> http://src.chromium.org/viewvc/native_client/trunk/src/native_client/pnacl/driver/pnacl-ld.py >>>> >>>> But, upon closer inspection, this looks like it's just messing around >>>> with the flags (splitting bitcode and native inputs?) and then calling out >>>> to other tools. Looks like it shells out to a "real" linker. Looks like >>>> maybe they use this: >>>> >>>> http://llvm.org/releases/3.0/docs/GoldPlugin.html >>>> >>>> (everything is very indirect, I'm only pretty sure this is what it >>>> does). >>>> >>>> Looks like this allows the system "gold" linker (a real linker that >>>> knows all about --start-group and friends) to interface with llvm in a way >>>> that it can do global optimizations on bitcode files (and bitcode >>>> archives) >>>> to generate - I think - a single optimized bitcode output. >>>> >>>> I have no idea whether it's feasible to use this in emscripten! >>>> >>>> Thanks, >>>> Ryan >>>> >>>> On Mon, Apr 14, 2014 at 2:26 PM, Alon Zakai <[email protected]>wrote: >>>> >>>>> That link doesn't work for me. But that makes sense. >>>>> >>>>> Can we perhaps reuse their code? What do they do to solve this, how do >>>>> they actually do the linking? llvm-link like us? >>>>> >>>>> - Alon >>>>> >>>>> >>>>> >>>>> On Thu, Apr 10, 2014 at 11:56 AM, Ryan Sturgell < >>>>> [email protected]> wrote: >>>>> >>>>>> Good point. Pnacl has an ld replacement called "pnacl-ld.py" where >>>>>> they handle stuff like --start-group,--end-group: >>>>>> >>>>>> >>>>>> https://code.google.com/p/chromium/codesearch#chromium/src/native_client/pnacl/driver/pnacl-ld.py&l=508 >>>>>> >>>>>> Ryan >>>>>> >>>>>> On Thu, Apr 10, 2014 at 11:42 AM, Alon Zakai <[email protected]>wrote: >>>>>> >>>>>>> Regarding llvm-ld, might be worth finding out how other >>>>>>> bitcode-using projects do linking, like PNaCl. They have to do >>>>>>> something >>>>>>> for this as well, I assume. >>>>>>> >>>>>>> - Alon >>>>>>> >>>>>>> >>>>>>> >>>>>>> On Mon, Apr 7, 2014 at 9:00 PM, Ryan Sturgell < >>>>>>> [email protected]> wrote: >>>>>>> >>>>>>>> >>>>>>>> On Monday, April 7, 2014 4:37:12 PM UTC-7, Alon Zakai wrote: >>>>>>>>> >>>>>>>>> Very interesting about msvc. However different behavior from >>>>>>>>> gcc/clang sounds worrying. But, if there is no perf downside (as Ben >>>>>>>>> just >>>>>>>>> asked) and no compatibility issues, might be worth considering. >>>>>>>>> >>>>>>>> >>>>>>>> The compatibility issue would be that builds that fail with "symbol >>>>>>>> not found" in gcc/clang might succeed in emcc. >>>>>>>> >>>>>>>> Well... there is one real case that could be a problem. If you >>>>>>>> implement a function which overrides something from a library_*.js >>>>>>>> (like, >>>>>>>> say, glCreateShader), for example: >>>>>>>> >>>>>>>> emcc main.o lib1.a lib2.a >>>>>>>> >>>>>>>> Where: >>>>>>>> main.o does NOT call glCreateShader >>>>>>>> lib1.a contains obj1.o, which implements glCreateShader (and >>>>>>>> nothing that main.o depends on, so is not included on the first pass) >>>>>>>> lib2.a contains obj2.o, which uses glCreateShader and implements >>>>>>>> a function that main calls (so that it IS picked up on the first pass) >>>>>>>> >>>>>>>> With the current implementation, obj1.o is not included in the >>>>>>>> link, and glCreateShader ends up binding to the implementation in >>>>>>>> library_gl.js (however that works). >>>>>>>> >>>>>>>> But with --rescan-libs, obj1.o would be picked up on the second >>>>>>>> scan, and the implementation there would be used. >>>>>>>> >>>>>>>> >>>>>>>>> If we do handle groups, we would need to do it in emcc and the >>>>>>>>> linking code in tools/shared. >>>>>>>>> >>>>>>>> >>>>>>>> Right, that's where I made the changes for --rescan-libs in my >>>>>>>> branch. >>>>>>>> >>>>>>>> >>>>>>>>> We do linking ourselves except for calling llvm-link which is very >>>>>>>>> low-level and does not support groups and such, last I checked. >>>>>>>>> >>>>>>>> >>>>>>>> Correct. llvm-ld DID handle --start-group/--end-group but llvm-ld >>>>>>>> was dropped in llvm 3.2 because it was incomplete and clang is where >>>>>>>> all >>>>>>>> that logic happens now. >>>>>>>> >>>>>>>> >>>>>>>>> Hopefully it wouldn't be too complex, but I don't know offhand. >>>>>>>>> >>>>>>>> >>>>>>>> As I mentioned above it's not _too_ bad (the changes in >>>>>>>> tools/shared.py would be very similar to what I have in my branch), >>>>>>>> the >>>>>>>> hard part is plumbing it all together since the grouping needs to >>>>>>>> propagate >>>>>>>> through from emcc to shared.py. >>>>>>>> >>>>>>>> I guess the cleanest way would be to unwrap -Wl,--X options to --X >>>>>>>> in the option parsing in emcc, keep these around in the file list, >>>>>>>> make >>>>>>>> sure to pass them through all the transformations / filters that >>>>>>>> happen in >>>>>>>> there, and pass those through to shared.building.link. >>>>>>>> >>>>>>>> Ryan >>>>>>>> >>>>>>>> >>>>>>>>> - Alon >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> On Mon, Apr 7, 2014 at 2:07 PM, Ryan Sturgell <[email protected] >>>>>>>>> > wrote: >>>>>>>>> >>>>>>>>>> I have a patch at https://github.com/rsturgell/emscripten/tree/ >>>>>>>>>> rescan_libs which adds a --rescan-libs flag to emcc which >>>>>>>>>> behaves as if -Wl,--start-group and -Wl,--end-group surround all the >>>>>>>>>> inputs. >>>>>>>>>> >>>>>>>>>> I have to say, though, I hate to add another way to do specify >>>>>>>>>> this... I think it would be better to just properly support groups, >>>>>>>>>> but >>>>>>>>>> this will be more complex. We'd have to either handle the flags in >>>>>>>>>> emcc and >>>>>>>>>> maybe pass some kind of tree through to the link function, or >>>>>>>>>> propagate >>>>>>>>>> -Wl,* linker flags through and handle the grouping in link. But both >>>>>>>>>> of >>>>>>>>>> these interact poorly with the processing that happens to these >>>>>>>>>> inputs in >>>>>>>>>> emcc. It will be a big change to make it all work. >>>>>>>>>> >>>>>>>>>> Another option which doesn't add complexity is go with >>>>>>>>>> --rescan-libs but default to true (!). This way emcc will just work >>>>>>>>>> with >>>>>>>>>> whatever archive order. This is exactly how msvc does it (from >>>>>>>>>> http://msdn.microsoft.com/en-us/library/hcce369f.aspx): >>>>>>>>>> >>>>>>>>>> Object files on the command line are processed in the order they >>>>>>>>>> appear on the command line. Libraries are searched in command line >>>>>>>>>> order as >>>>>>>>>> well, with the following caveat: Symbols that are unresolved when >>>>>>>>>> bringing >>>>>>>>>> in an object file from a library are searched for in that library >>>>>>>>>> first, >>>>>>>>>> and then the following libraries from the command line and >>>>>>>>>> /DEFAULTLIB >>>>>>>>>> (Specify Default >>>>>>>>>> Library)<http://msdn.microsoft.com/en-us/library/229a6ysd.aspx> >>>>>>>>>> directives, >>>>>>>>>> and then to any libraries at the beginning of the command line. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Thoughts? >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> Ryan >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On Tuesday, April 1, 2014 6:48:28 AM UTC-7, Warren Seine wrote: >>>>>>>>>>> >>>>>>>>>>> This is something I need and it suits my requirements. At the >>>>>>>>>>> moment, I'm using the wonderful solution of adding libraries twice >>>>>>>>>>> in the >>>>>>>>>>> command line (which works btw...), so anything cleaner is better. >>>>>>>>>>> >>>>>>>>>>> On Tuesday, April 1, 2014 5:27:24 AM UTC+2, Alon Zakai wrote: >>>>>>>>>>>> >>>>>>>>>>>> From the perspective of maintainability both sound reasonable, >>>>>>>>>>>> the latter sounds much simpler though, so if that is good enough >>>>>>>>>>>> for the >>>>>>>>>>>> people that want this functionality then it is preferable. >>>>>>>>>>>> >>>>>>>>>>>> - Alon >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> On Fri, Mar 28, 2014 at 2:24 PM, Ryan Sturgell < >>>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>>> >>>>>>>>>>>>> Hi all, I'm also interested in --start-group/--end-group. I'm >>>>>>>>>>>>> trying to build an existing project which uses gyp and ninja to >>>>>>>>>>>>> build, and >>>>>>>>>>>>> gyp basically passes the .a files in an arbitrary order and wraps >>>>>>>>>>>>> the whole >>>>>>>>>>>>> set in --start-group/--end-group. >>>>>>>>>>>>> >>>>>>>>>>>>> Supporting this functionality in shared.Building.link should >>>>>>>>>>>>> be possible (passing a list of archive-groups instead of a list >>>>>>>>>>>>> of >>>>>>>>>>>>> archives). >>>>>>>>>>>>> >>>>>>>>>>>>> Alternatively, the particular case of gyp could be handled by >>>>>>>>>>>>> supporting a new emcc-specific flag like --group-all or something >>>>>>>>>>>>> which >>>>>>>>>>>>> specifies that the whole set of archives should be repeatedly >>>>>>>>>>>>> scanned as if >>>>>>>>>>>>> they were all contained in --start-group/--end-group. This would >>>>>>>>>>>>> be simpler >>>>>>>>>>>>> to implement. >>>>>>>>>>>>> >>>>>>>>>>>>> Do either of these options sound palatable? I could take a >>>>>>>>>>>>> stab and putting a patch together. >>>>>>>>>>>>> >>>>>>>>>>>>> Thanks, >>>>>>>>>>>>> Ryan >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> On Thursday, March 6, 2014 3:18:27 PM UTC-8, Alon Zakai wrote: >>>>>>>>>>>>> >>>>>>>>>>>>>> Hmm yes, .a files do work that way I believe, and we support >>>>>>>>>>>>>> linking them with proper semantics, so that should work. >>>>>>>>>>>>>> >>>>>>>>>>>>>> - Alon >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> On Thu, Mar 6, 2014 at 12:26 AM, Warren Seine < >>>>>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>>>>> >>>>>>>>>>>>>>> Thank you. I've had a look at the llvm-link source code, and >>>>>>>>>>>>>>> there used to be an option --start-group which was simply >>>>>>>>>>>>>>> ignored (and has since been withdrawn), so I'm wondering if >>>>>>>>>>>>>>> that's the >>>>>>>>>>>>>>> default behaviour now. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Anyway, somebody mentioned that I could put together all the >>>>>>>>>>>>>>> .a files into a single one and then only link the big one. I >>>>>>>>>>>>>>> guess that'd >>>>>>>>>>>>>>> work? I'll try that. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> On Wednesday, March 5, 2014 10:56:42 PM UTC+1, Alon Zakai >>>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> The most relevant doc is likely https://github.com/kripken/ >>>>>>>>>>>>>>>> emscripten/wiki/Building-Projects but you can just look in >>>>>>>>>>>>>>>> emcc to see what we support and do not support. I've never >>>>>>>>>>>>>>>> heard of that >>>>>>>>>>>>>>>> option, so I guess we don't support it. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> - Alon >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> On Wed, Mar 5, 2014 at 12:50 AM, Warren Seine < >>>>>>>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> @Alon, any idea? Can you describe (or point me to a page >>>>>>>>>>>>>>>>> explaining) the linking step? >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> On Tuesday, March 4, 2014 9:21:09 AM UTC+1, Warren Seine >>>>>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Hi, >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> -Wl,--start-group and -Wl,--end-group are compiler >>>>>>>>>>>>>>>>>> options to tell the linker to look for symbols in previous >>>>>>>>>>>>>>>>>> libraries, as in >>>>>>>>>>>>>>>>>> "before in the command line". They seem to have no effect on >>>>>>>>>>>>>>>>>> link order >>>>>>>>>>>>>>>>>> with emcc and I'm not really surprised because the link >>>>>>>>>>>>>>>>>> step is different from native builds in many ways. I'm >>>>>>>>>>>>>>>>>> wondering if someone >>>>>>>>>>>>>>>>>> has been there and found a workaround to avoid unresolved >>>>>>>>>>>>>>>>>> symbols when >>>>>>>>>>>>>>>>>> linking static libraries with circular dependencies. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> I can reorder manually reorder libraries or put them >>>>>>>>>>>>>>>>>> twice, but for some reason I'd prefer to use this feature >>>>>>>>>>>>>>>>>> (or something >>>>>>>>>>>>>>>>>> similar) which works fine with gcc. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Cheers, >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>> -- >>>>>>>>>> You received this message because you are subscribed to the >>>>>>>>>> Google Groups "emscripten-discuss" group. >>>>>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>>>>> send an email to [email protected]. >>>>>>>>>> For more options, visit https://groups.google.com/d/optout. >>>>>>>>>> >>>>>>>>> >>>>>>>>> -- >>>>>>>> You received this message because you are subscribed to the Google >>>>>>>> Groups "emscripten-discuss" group. >>>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>>> send an email to [email protected]. >>>>>>>> For more options, visit https://groups.google.com/d/optout. >>>>>>>> >>>>>>> >>>>>>> -- >>>>>>> You received this message because you are subscribed to a topic in >>>>>>> the Google Groups "emscripten-discuss" group. >>>>>>> To unsubscribe from this topic, visit >>>>>>> https://groups.google.com/d/topic/emscripten-discuss/n1qfKPPAy08/unsubscribe >>>>>>> . >>>>>>> To unsubscribe from this group and all its topics, send an email to >>>>>>> [email protected]. >>>>>>> >>>>>>> For more options, visit https://groups.google.com/d/optout. >>>>>>> >>>>>> >>>>>> -- >>>>>> You received this message because you are subscribed to the Google >>>>>> Groups "emscripten-discuss" group. >>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>> send an email to [email protected]. >>>>>> For more options, visit https://groups.google.com/d/optout. >>>>>> >>>>> >>>>> -- >>>>> You received this message because you are subscribed to a topic in the >>>>> Google Groups "emscripten-discuss" group. >>>>> To unsubscribe from this topic, visit >>>>> https://groups.google.com/d/topic/emscripten-discuss/n1qfKPPAy08/unsubscribe >>>>> . >>>>> To unsubscribe from this group and all its topics, send an email to >>>>> [email protected]. >>>>> For more options, visit https://groups.google.com/d/optout. >>>>> >>>> >>>> -- >>>> You received this message because you are subscribed to the Google >>>> Groups "emscripten-discuss" group. >>>> To unsubscribe from this group and stop receiving emails from it, send >>>> an email to [email protected]. >>>> For more options, visit https://groups.google.com/d/optout. >>>> >>> >>> >> -- >> You received this message because you are subscribed to a topic in the >> Google Groups "emscripten-discuss" group. >> To unsubscribe from this topic, visit >> https://groups.google.com/d/topic/emscripten-discuss/n1qfKPPAy08/unsubscribe >> . >> To unsubscribe from this group and all its topics, send an email to >> [email protected]. >> For more options, visit https://groups.google.com/d/optout. >> > > -- You received this message because you are subscribed to the Google Groups "emscripten-discuss" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
