Agreed. I have a pretty clean change to support --start/end-group, I'll send you a pull request.
Ryan On Tue, Apr 22, 2014 at 10:17 AM, Alon Zakai <[email protected]> wrote: > Regardless though, it would be a lot of work for us to switch to a gold > plugin, both development and maintenance, so I doubt we would do it, as the > only missing feature I am aware of is the groups issue being discussed > here. We should fix that directly I think. > > - Alon > > > > On Tue, Apr 22, 2014 at 10:16 AM, Alon Zakai <[email protected]> wrote: > >> Thanks for looking into this, I'll ask some pnacl people I know. >> >> - Alon >> >> >> >> On Mon, Apr 14, 2014 at 4:15 PM, Ryan Sturgell >> <[email protected]>wrote: >> >>> Hmm, yes, that file seems to have disappeared from that location! Let's >>> try again: >>> >>> >>> http://src.chromium.org/viewvc/native_client/trunk/src/native_client/pnacl/driver/pnacl-ld.py >>> >>> But, upon closer inspection, this looks like it's just messing around >>> with the flags (splitting bitcode and native inputs?) and then calling out >>> to other tools. Looks like it shells out to a "real" linker. Looks like >>> maybe they use this: >>> >>> http://llvm.org/releases/3.0/docs/GoldPlugin.html >>> >>> (everything is very indirect, I'm only pretty sure this is what it does). >>> >>> Looks like this allows the system "gold" linker (a real linker that >>> knows all about --start-group and friends) to interface with llvm in a way >>> that it can do global optimizations on bitcode files (and bitcode archives) >>> to generate - I think - a single optimized bitcode output. >>> >>> I have no idea whether it's feasible to use this in emscripten! >>> >>> Thanks, >>> Ryan >>> >>> On Mon, Apr 14, 2014 at 2:26 PM, Alon Zakai <[email protected]> wrote: >>> >>>> That link doesn't work for me. But that makes sense. >>>> >>>> Can we perhaps reuse their code? What do they do to solve this, how do >>>> they actually do the linking? llvm-link like us? >>>> >>>> - Alon >>>> >>>> >>>> >>>> On Thu, Apr 10, 2014 at 11:56 AM, Ryan Sturgell < >>>> [email protected]> wrote: >>>> >>>>> Good point. Pnacl has an ld replacement called "pnacl-ld.py" where >>>>> they handle stuff like --start-group,--end-group: >>>>> >>>>> >>>>> https://code.google.com/p/chromium/codesearch#chromium/src/native_client/pnacl/driver/pnacl-ld.py&l=508 >>>>> >>>>> Ryan >>>>> >>>>> On Thu, Apr 10, 2014 at 11:42 AM, Alon Zakai <[email protected]>wrote: >>>>> >>>>>> Regarding llvm-ld, might be worth finding out how other bitcode-using >>>>>> projects do linking, like PNaCl. They have to do something for this as >>>>>> well, I assume. >>>>>> >>>>>> - Alon >>>>>> >>>>>> >>>>>> >>>>>> On Mon, Apr 7, 2014 at 9:00 PM, Ryan Sturgell < >>>>>> [email protected]> wrote: >>>>>> >>>>>>> >>>>>>> On Monday, April 7, 2014 4:37:12 PM UTC-7, Alon Zakai wrote: >>>>>>>> >>>>>>>> Very interesting about msvc. However different behavior from >>>>>>>> gcc/clang sounds worrying. But, if there is no perf downside (as Ben >>>>>>>> just >>>>>>>> asked) and no compatibility issues, might be worth considering. >>>>>>>> >>>>>>> >>>>>>> The compatibility issue would be that builds that fail with "symbol >>>>>>> not found" in gcc/clang might succeed in emcc. >>>>>>> >>>>>>> Well... there is one real case that could be a problem. If you >>>>>>> implement a function which overrides something from a library_*.js >>>>>>> (like, >>>>>>> say, glCreateShader), for example: >>>>>>> >>>>>>> emcc main.o lib1.a lib2.a >>>>>>> >>>>>>> Where: >>>>>>> main.o does NOT call glCreateShader >>>>>>> lib1.a contains obj1.o, which implements glCreateShader (and >>>>>>> nothing that main.o depends on, so is not included on the first pass) >>>>>>> lib2.a contains obj2.o, which uses glCreateShader and implements a >>>>>>> function that main calls (so that it IS picked up on the first pass) >>>>>>> >>>>>>> With the current implementation, obj1.o is not included in the link, >>>>>>> and glCreateShader ends up binding to the implementation in >>>>>>> library_gl.js >>>>>>> (however that works). >>>>>>> >>>>>>> But with --rescan-libs, obj1.o would be picked up on the second >>>>>>> scan, and the implementation there would be used. >>>>>>> >>>>>>> >>>>>>>> If we do handle groups, we would need to do it in emcc and the >>>>>>>> linking code in tools/shared. >>>>>>>> >>>>>>> >>>>>>> Right, that's where I made the changes for --rescan-libs in my >>>>>>> branch. >>>>>>> >>>>>>> >>>>>>>> We do linking ourselves except for calling llvm-link which is very >>>>>>>> low-level and does not support groups and such, last I checked. >>>>>>>> >>>>>>> >>>>>>> Correct. llvm-ld DID handle --start-group/--end-group but llvm-ld >>>>>>> was dropped in llvm 3.2 because it was incomplete and clang is where all >>>>>>> that logic happens now. >>>>>>> >>>>>>> >>>>>>>> Hopefully it wouldn't be too complex, but I don't know offhand. >>>>>>>> >>>>>>> >>>>>>> As I mentioned above it's not _too_ bad (the changes in >>>>>>> tools/shared.py would be very similar to what I have in my branch), the >>>>>>> hard part is plumbing it all together since the grouping needs to >>>>>>> propagate >>>>>>> through from emcc to shared.py. >>>>>>> >>>>>>> I guess the cleanest way would be to unwrap -Wl,--X options to --X >>>>>>> in the option parsing in emcc, keep these around in the file list, make >>>>>>> sure to pass them through all the transformations / filters that happen >>>>>>> in >>>>>>> there, and pass those through to shared.building.link. >>>>>>> >>>>>>> Ryan >>>>>>> >>>>>>> >>>>>>>> - Alon >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> On Mon, Apr 7, 2014 at 2:07 PM, Ryan Sturgell >>>>>>>> <[email protected]>wrote: >>>>>>>> >>>>>>>>> I have a patch at https://github.com/rsturgell/emscripten/tree/ >>>>>>>>> rescan_libs which adds a --rescan-libs flag to emcc which behaves >>>>>>>>> as if -Wl,--start-group and -Wl,--end-group surround all the inputs. >>>>>>>>> >>>>>>>>> I have to say, though, I hate to add another way to do specify >>>>>>>>> this... I think it would be better to just properly support groups, >>>>>>>>> but >>>>>>>>> this will be more complex. We'd have to either handle the flags in >>>>>>>>> emcc and >>>>>>>>> maybe pass some kind of tree through to the link function, or >>>>>>>>> propagate >>>>>>>>> -Wl,* linker flags through and handle the grouping in link. But both >>>>>>>>> of >>>>>>>>> these interact poorly with the processing that happens to these >>>>>>>>> inputs in >>>>>>>>> emcc. It will be a big change to make it all work. >>>>>>>>> >>>>>>>>> Another option which doesn't add complexity is go with >>>>>>>>> --rescan-libs but default to true (!). This way emcc will just work >>>>>>>>> with >>>>>>>>> whatever archive order. This is exactly how msvc does it (from >>>>>>>>> http://msdn.microsoft.com/en-us/library/hcce369f.aspx): >>>>>>>>> >>>>>>>>> Object files on the command line are processed in the order they >>>>>>>>> appear on the command line. Libraries are searched in command line >>>>>>>>> order as >>>>>>>>> well, with the following caveat: Symbols that are unresolved when >>>>>>>>> bringing >>>>>>>>> in an object file from a library are searched for in that library >>>>>>>>> first, >>>>>>>>> and then the following libraries from the command line and /DEFAULTLIB >>>>>>>>> (Specify Default >>>>>>>>> Library)<http://msdn.microsoft.com/en-us/library/229a6ysd.aspx> >>>>>>>>> directives, >>>>>>>>> and then to any libraries at the beginning of the command line. >>>>>>>>> >>>>>>>>> >>>>>>>>> Thoughts? >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> Ryan >>>>>>>>> >>>>>>>>> >>>>>>>>> On Tuesday, April 1, 2014 6:48:28 AM UTC-7, Warren Seine wrote: >>>>>>>>>> >>>>>>>>>> This is something I need and it suits my requirements. At the >>>>>>>>>> moment, I'm using the wonderful solution of adding libraries twice >>>>>>>>>> in the >>>>>>>>>> command line (which works btw...), so anything cleaner is better. >>>>>>>>>> >>>>>>>>>> On Tuesday, April 1, 2014 5:27:24 AM UTC+2, Alon Zakai wrote: >>>>>>>>>>> >>>>>>>>>>> From the perspective of maintainability both sound reasonable, >>>>>>>>>>> the latter sounds much simpler though, so if that is good enough >>>>>>>>>>> for the >>>>>>>>>>> people that want this functionality then it is preferable. >>>>>>>>>>> >>>>>>>>>>> - Alon >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On Fri, Mar 28, 2014 at 2:24 PM, Ryan Sturgell < >>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>> >>>>>>>>>>>> Hi all, I'm also interested in --start-group/--end-group. I'm >>>>>>>>>>>> trying to build an existing project which uses gyp and ninja to >>>>>>>>>>>> build, and >>>>>>>>>>>> gyp basically passes the .a files in an arbitrary order and wraps >>>>>>>>>>>> the whole >>>>>>>>>>>> set in --start-group/--end-group. >>>>>>>>>>>> >>>>>>>>>>>> Supporting this functionality in shared.Building.link should be >>>>>>>>>>>> possible (passing a list of archive-groups instead of a list of >>>>>>>>>>>> archives). >>>>>>>>>>>> >>>>>>>>>>>> Alternatively, the particular case of gyp could be handled by >>>>>>>>>>>> supporting a new emcc-specific flag like --group-all or something >>>>>>>>>>>> which >>>>>>>>>>>> specifies that the whole set of archives should be repeatedly >>>>>>>>>>>> scanned as if >>>>>>>>>>>> they were all contained in --start-group/--end-group. This would >>>>>>>>>>>> be simpler >>>>>>>>>>>> to implement. >>>>>>>>>>>> >>>>>>>>>>>> Do either of these options sound palatable? I could take a stab >>>>>>>>>>>> and putting a patch together. >>>>>>>>>>>> >>>>>>>>>>>> Thanks, >>>>>>>>>>>> Ryan >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> On Thursday, March 6, 2014 3:18:27 PM UTC-8, Alon Zakai wrote: >>>>>>>>>>>> >>>>>>>>>>>>> Hmm yes, .a files do work that way I believe, and we support >>>>>>>>>>>>> linking them with proper semantics, so that should work. >>>>>>>>>>>>> >>>>>>>>>>>>> - Alon >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> On Thu, Mar 6, 2014 at 12:26 AM, Warren Seine < >>>>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>>>> >>>>>>>>>>>>>> Thank you. I've had a look at the llvm-link source code, and >>>>>>>>>>>>>> there used to be an option --start-group which was simply >>>>>>>>>>>>>> ignored (and has since been withdrawn), so I'm wondering if >>>>>>>>>>>>>> that's the >>>>>>>>>>>>>> default behaviour now. >>>>>>>>>>>>>> >>>>>>>>>>>>>> Anyway, somebody mentioned that I could put together all the >>>>>>>>>>>>>> .a files into a single one and then only link the big one. I >>>>>>>>>>>>>> guess that'd >>>>>>>>>>>>>> work? I'll try that. >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> On Wednesday, March 5, 2014 10:56:42 PM UTC+1, Alon Zakai >>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>> >>>>>>>>>>>>>>> The most relevant doc is likely https://github.com/kripken/ >>>>>>>>>>>>>>> emscripten/wiki/Building-Projects but you can just look in >>>>>>>>>>>>>>> emcc to see what we support and do not support. I've never >>>>>>>>>>>>>>> heard of that >>>>>>>>>>>>>>> option, so I guess we don't support it. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> - Alon >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> On Wed, Mar 5, 2014 at 12:50 AM, Warren Seine < >>>>>>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> @Alon, any idea? Can you describe (or point me to a page >>>>>>>>>>>>>>>> explaining) the linking step? >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> On Tuesday, March 4, 2014 9:21:09 AM UTC+1, Warren Seine >>>>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Hi, >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> -Wl,--start-group and -Wl,--end-group are compiler >>>>>>>>>>>>>>>>> options to tell the linker to look for symbols in previous >>>>>>>>>>>>>>>>> libraries, as in >>>>>>>>>>>>>>>>> "before in the command line". They seem to have no effect on >>>>>>>>>>>>>>>>> link order >>>>>>>>>>>>>>>>> with emcc and I'm not really surprised because the link >>>>>>>>>>>>>>>>> step is different from native builds in many ways. I'm >>>>>>>>>>>>>>>>> wondering if someone >>>>>>>>>>>>>>>>> has been there and found a workaround to avoid unresolved >>>>>>>>>>>>>>>>> symbols when >>>>>>>>>>>>>>>>> linking static libraries with circular dependencies. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> I can reorder manually reorder libraries or put them >>>>>>>>>>>>>>>>> twice, but for some reason I'd prefer to use this feature (or >>>>>>>>>>>>>>>>> something >>>>>>>>>>>>>>>>> similar) which works fine with gcc. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Cheers, >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>> -- >>>>>>>>> You received this message because you are subscribed to the Google >>>>>>>>> Groups "emscripten-discuss" group. >>>>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>>>> send an email to [email protected]. >>>>>>>>> For more options, visit https://groups.google.com/d/optout. >>>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>> You received this message because you are subscribed to the Google >>>>>>> Groups "emscripten-discuss" group. >>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>> send an email to [email protected]. >>>>>>> For more options, visit https://groups.google.com/d/optout. >>>>>>> >>>>>> >>>>>> -- >>>>>> You received this message because you are subscribed to a topic in >>>>>> the Google Groups "emscripten-discuss" group. >>>>>> To unsubscribe from this topic, visit >>>>>> https://groups.google.com/d/topic/emscripten-discuss/n1qfKPPAy08/unsubscribe >>>>>> . >>>>>> To unsubscribe from this group and all its topics, send an email to >>>>>> [email protected]. >>>>>> >>>>>> For more options, visit https://groups.google.com/d/optout. >>>>>> >>>>> >>>>> -- >>>>> You received this message because you are subscribed to the Google >>>>> Groups "emscripten-discuss" group. >>>>> To unsubscribe from this group and stop receiving emails from it, send >>>>> an email to [email protected]. >>>>> For more options, visit https://groups.google.com/d/optout. >>>>> >>>> >>>> -- >>>> You received this message because you are subscribed to a topic in the >>>> Google Groups "emscripten-discuss" group. >>>> To unsubscribe from this topic, visit >>>> https://groups.google.com/d/topic/emscripten-discuss/n1qfKPPAy08/unsubscribe >>>> . >>>> To unsubscribe from this group and all its topics, send an email to >>>> [email protected]. >>>> For more options, visit https://groups.google.com/d/optout. >>>> >>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "emscripten-discuss" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to [email protected]. >>> For more options, visit https://groups.google.com/d/optout. >>> >> >> > -- > You received this message because you are subscribed to a topic in the > Google Groups "emscripten-discuss" group. > To unsubscribe from this topic, visit > https://groups.google.com/d/topic/emscripten-discuss/n1qfKPPAy08/unsubscribe > . > To unsubscribe from this group and all its topics, send an email to > [email protected]. > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "emscripten-discuss" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
