Thanks for looking into this, I'll ask some pnacl people I know. - Alon
On Mon, Apr 14, 2014 at 4:15 PM, Ryan Sturgell <[email protected]>wrote: > Hmm, yes, that file seems to have disappeared from that location! Let's > try again: > > > http://src.chromium.org/viewvc/native_client/trunk/src/native_client/pnacl/driver/pnacl-ld.py > > But, upon closer inspection, this looks like it's just messing around with > the flags (splitting bitcode and native inputs?) and then calling out to > other tools. Looks like it shells out to a "real" linker. Looks like maybe > they use this: > > http://llvm.org/releases/3.0/docs/GoldPlugin.html > > (everything is very indirect, I'm only pretty sure this is what it does). > > Looks like this allows the system "gold" linker (a real linker that knows > all about --start-group and friends) to interface with llvm in a way that > it can do global optimizations on bitcode files (and bitcode archives) to > generate - I think - a single optimized bitcode output. > > I have no idea whether it's feasible to use this in emscripten! > > Thanks, > Ryan > > On Mon, Apr 14, 2014 at 2:26 PM, Alon Zakai <[email protected]> wrote: > >> That link doesn't work for me. But that makes sense. >> >> Can we perhaps reuse their code? What do they do to solve this, how do >> they actually do the linking? llvm-link like us? >> >> - Alon >> >> >> >> On Thu, Apr 10, 2014 at 11:56 AM, Ryan Sturgell >> <[email protected]>wrote: >> >>> Good point. Pnacl has an ld replacement called "pnacl-ld.py" where they >>> handle stuff like --start-group,--end-group: >>> >>> >>> https://code.google.com/p/chromium/codesearch#chromium/src/native_client/pnacl/driver/pnacl-ld.py&l=508 >>> >>> Ryan >>> >>> On Thu, Apr 10, 2014 at 11:42 AM, Alon Zakai <[email protected]>wrote: >>> >>>> Regarding llvm-ld, might be worth finding out how other bitcode-using >>>> projects do linking, like PNaCl. They have to do something for this as >>>> well, I assume. >>>> >>>> - Alon >>>> >>>> >>>> >>>> On Mon, Apr 7, 2014 at 9:00 PM, Ryan Sturgell >>>> <[email protected]>wrote: >>>> >>>>> >>>>> On Monday, April 7, 2014 4:37:12 PM UTC-7, Alon Zakai wrote: >>>>>> >>>>>> Very interesting about msvc. However different behavior from >>>>>> gcc/clang sounds worrying. But, if there is no perf downside (as Ben just >>>>>> asked) and no compatibility issues, might be worth considering. >>>>>> >>>>> >>>>> The compatibility issue would be that builds that fail with "symbol >>>>> not found" in gcc/clang might succeed in emcc. >>>>> >>>>> Well... there is one real case that could be a problem. If you >>>>> implement a function which overrides something from a library_*.js (like, >>>>> say, glCreateShader), for example: >>>>> >>>>> emcc main.o lib1.a lib2.a >>>>> >>>>> Where: >>>>> main.o does NOT call glCreateShader >>>>> lib1.a contains obj1.o, which implements glCreateShader (and >>>>> nothing that main.o depends on, so is not included on the first pass) >>>>> lib2.a contains obj2.o, which uses glCreateShader and implements a >>>>> function that main calls (so that it IS picked up on the first pass) >>>>> >>>>> With the current implementation, obj1.o is not included in the link, >>>>> and glCreateShader ends up binding to the implementation in library_gl.js >>>>> (however that works). >>>>> >>>>> But with --rescan-libs, obj1.o would be picked up on the second scan, >>>>> and the implementation there would be used. >>>>> >>>>> >>>>>> If we do handle groups, we would need to do it in emcc and the >>>>>> linking code in tools/shared. >>>>>> >>>>> >>>>> Right, that's where I made the changes for --rescan-libs in my branch. >>>>> >>>>> >>>>>> We do linking ourselves except for calling llvm-link which is very >>>>>> low-level and does not support groups and such, last I checked. >>>>>> >>>>> >>>>> Correct. llvm-ld DID handle --start-group/--end-group but llvm-ld was >>>>> dropped in llvm 3.2 because it was incomplete and clang is where all that >>>>> logic happens now. >>>>> >>>>> >>>>>> Hopefully it wouldn't be too complex, but I don't know offhand. >>>>>> >>>>> >>>>> As I mentioned above it's not _too_ bad (the changes in >>>>> tools/shared.py would be very similar to what I have in my branch), the >>>>> hard part is plumbing it all together since the grouping needs to >>>>> propagate >>>>> through from emcc to shared.py. >>>>> >>>>> I guess the cleanest way would be to unwrap -Wl,--X options to --X in >>>>> the option parsing in emcc, keep these around in the file list, make sure >>>>> to pass them through all the transformations / filters that happen in >>>>> there, and pass those through to shared.building.link. >>>>> >>>>> Ryan >>>>> >>>>> >>>>>> - Alon >>>>>> >>>>>> >>>>>> >>>>>> On Mon, Apr 7, 2014 at 2:07 PM, Ryan Sturgell <[email protected]>wrote: >>>>>> >>>>>>> I have a patch at https://github.com/rsturgell/emscripten/tree/ >>>>>>> rescan_libs which adds a --rescan-libs flag to emcc which behaves >>>>>>> as if -Wl,--start-group and -Wl,--end-group surround all the inputs. >>>>>>> >>>>>>> I have to say, though, I hate to add another way to do specify >>>>>>> this... I think it would be better to just properly support groups, but >>>>>>> this will be more complex. We'd have to either handle the flags in emcc >>>>>>> and >>>>>>> maybe pass some kind of tree through to the link function, or propagate >>>>>>> -Wl,* linker flags through and handle the grouping in link. But both of >>>>>>> these interact poorly with the processing that happens to these inputs >>>>>>> in >>>>>>> emcc. It will be a big change to make it all work. >>>>>>> >>>>>>> Another option which doesn't add complexity is go with --rescan-libs >>>>>>> but default to true (!). This way emcc will just work with whatever >>>>>>> archive >>>>>>> order. This is exactly how msvc does it (from >>>>>>> http://msdn.microsoft.com/en-us/library/hcce369f.aspx): >>>>>>> >>>>>>> Object files on the command line are processed in the order they >>>>>>> appear on the command line. Libraries are searched in command line >>>>>>> order as >>>>>>> well, with the following caveat: Symbols that are unresolved when >>>>>>> bringing >>>>>>> in an object file from a library are searched for in that library first, >>>>>>> and then the following libraries from the command line and /DEFAULTLIB >>>>>>> (Specify Default >>>>>>> Library)<http://msdn.microsoft.com/en-us/library/229a6ysd.aspx> >>>>>>> directives, >>>>>>> and then to any libraries at the beginning of the command line. >>>>>>> >>>>>>> >>>>>>> Thoughts? >>>>>>> >>>>>>> Thanks, >>>>>>> Ryan >>>>>>> >>>>>>> >>>>>>> On Tuesday, April 1, 2014 6:48:28 AM UTC-7, Warren Seine wrote: >>>>>>>> >>>>>>>> This is something I need and it suits my requirements. At the >>>>>>>> moment, I'm using the wonderful solution of adding libraries twice in >>>>>>>> the >>>>>>>> command line (which works btw...), so anything cleaner is better. >>>>>>>> >>>>>>>> On Tuesday, April 1, 2014 5:27:24 AM UTC+2, Alon Zakai wrote: >>>>>>>>> >>>>>>>>> From the perspective of maintainability both sound reasonable, the >>>>>>>>> latter sounds much simpler though, so if that is good enough for the >>>>>>>>> people >>>>>>>>> that want this functionality then it is preferable. >>>>>>>>> >>>>>>>>> - Alon >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> On Fri, Mar 28, 2014 at 2:24 PM, Ryan Sturgell < >>>>>>>>> [email protected]> wrote: >>>>>>>>> >>>>>>>>>> Hi all, I'm also interested in --start-group/--end-group. I'm >>>>>>>>>> trying to build an existing project which uses gyp and ninja to >>>>>>>>>> build, and >>>>>>>>>> gyp basically passes the .a files in an arbitrary order and wraps >>>>>>>>>> the whole >>>>>>>>>> set in --start-group/--end-group. >>>>>>>>>> >>>>>>>>>> Supporting this functionality in shared.Building.link should be >>>>>>>>>> possible (passing a list of archive-groups instead of a list of >>>>>>>>>> archives). >>>>>>>>>> >>>>>>>>>> Alternatively, the particular case of gyp could be handled by >>>>>>>>>> supporting a new emcc-specific flag like --group-all or something >>>>>>>>>> which >>>>>>>>>> specifies that the whole set of archives should be repeatedly >>>>>>>>>> scanned as if >>>>>>>>>> they were all contained in --start-group/--end-group. This would be >>>>>>>>>> simpler >>>>>>>>>> to implement. >>>>>>>>>> >>>>>>>>>> Do either of these options sound palatable? I could take a stab >>>>>>>>>> and putting a patch together. >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> Ryan >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On Thursday, March 6, 2014 3:18:27 PM UTC-8, Alon Zakai wrote: >>>>>>>>>> >>>>>>>>>>> Hmm yes, .a files do work that way I believe, and we support >>>>>>>>>>> linking them with proper semantics, so that should work. >>>>>>>>>>> >>>>>>>>>>> - Alon >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On Thu, Mar 6, 2014 at 12:26 AM, Warren Seine < >>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>> >>>>>>>>>>>> Thank you. I've had a look at the llvm-link source code, and >>>>>>>>>>>> there used to be an option --start-group which was simply >>>>>>>>>>>> ignored (and has since been withdrawn), so I'm wondering if that's >>>>>>>>>>>> the >>>>>>>>>>>> default behaviour now. >>>>>>>>>>>> >>>>>>>>>>>> Anyway, somebody mentioned that I could put together all the .a >>>>>>>>>>>> files into a single one and then only link the big one. I guess >>>>>>>>>>>> that'd >>>>>>>>>>>> work? I'll try that. >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> On Wednesday, March 5, 2014 10:56:42 PM UTC+1, Alon Zakai wrote: >>>>>>>>>>>> >>>>>>>>>>>>> The most relevant doc is likely https://github.com/kripken/ems >>>>>>>>>>>>> cripten/wiki/Building-Projects but you can just look in emcc >>>>>>>>>>>>> to see what we support and do not support. I've never heard of >>>>>>>>>>>>> that option, >>>>>>>>>>>>> so I guess we don't support it. >>>>>>>>>>>>> >>>>>>>>>>>>> - Alon >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> On Wed, Mar 5, 2014 at 12:50 AM, Warren Seine < >>>>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>>>> >>>>>>>>>>>>>> @Alon, any idea? Can you describe (or point me to a page >>>>>>>>>>>>>> explaining) the linking step? >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> On Tuesday, March 4, 2014 9:21:09 AM UTC+1, Warren Seine >>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Hi, >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> -Wl,--start-group and -Wl,--end-group are compiler options >>>>>>>>>>>>>>> to tell the linker to look for symbols in previous libraries, >>>>>>>>>>>>>>> as in "before >>>>>>>>>>>>>>> in the command line". They seem to have no effect on link order >>>>>>>>>>>>>>> with >>>>>>>>>>>>>>> emcc and I'm not really surprised because the link step is >>>>>>>>>>>>>>> different from native builds in many ways. I'm wondering if >>>>>>>>>>>>>>> someone has >>>>>>>>>>>>>>> been there and found a workaround to avoid unresolved symbols >>>>>>>>>>>>>>> when linking >>>>>>>>>>>>>>> static libraries with circular dependencies. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> I can reorder manually reorder libraries or put them twice, >>>>>>>>>>>>>>> but for some reason I'd prefer to use this feature (or >>>>>>>>>>>>>>> something similar) >>>>>>>>>>>>>>> which works fine with gcc. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Cheers, >>>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>> -- >>>>>>> You received this message because you are subscribed to the Google >>>>>>> Groups "emscripten-discuss" group. >>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>> send an email to [email protected]. >>>>>>> For more options, visit https://groups.google.com/d/optout. >>>>>>> >>>>>> >>>>>> -- >>>>> You received this message because you are subscribed to the Google >>>>> Groups "emscripten-discuss" group. >>>>> To unsubscribe from this group and stop receiving emails from it, send >>>>> an email to [email protected]. >>>>> For more options, visit https://groups.google.com/d/optout. >>>>> >>>> >>>> -- >>>> You received this message because you are subscribed to a topic in the >>>> Google Groups "emscripten-discuss" group. >>>> To unsubscribe from this topic, visit >>>> https://groups.google.com/d/topic/emscripten-discuss/n1qfKPPAy08/unsubscribe >>>> . >>>> To unsubscribe from this group and all its topics, send an email to >>>> [email protected]. >>>> >>>> For more options, visit https://groups.google.com/d/optout. >>>> >>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "emscripten-discuss" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to [email protected]. >>> For more options, visit https://groups.google.com/d/optout. >>> >> >> -- >> You received this message because you are subscribed to a topic in the >> Google Groups "emscripten-discuss" group. >> To unsubscribe from this topic, visit >> https://groups.google.com/d/topic/emscripten-discuss/n1qfKPPAy08/unsubscribe >> . >> To unsubscribe from this group and all its topics, send an email to >> [email protected]. >> For more options, visit https://groups.google.com/d/optout. >> > > -- > You received this message because you are subscribed to the Google Groups > "emscripten-discuss" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "emscripten-discuss" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
