Re: Resolving LTO symbols for static library boundary
On Dienstag, 6. Februar 2018 01:01:29 CET Jan Hubicka wrote: > Dne 2018-02-05 18:44, Richard Biener napsal: > > On February 5, 2018 12:26:58 PM GMT+01:00, Allan Sandfeld Jensen > > > >wrote: > >> Hello GCC > >> > >> In trying to make it possible to use LTO for distro-builds of Qt, I > >> have again > >> hit the problem of static libraries. In Qt in general we for LTO rely > >> on a > >> library boundary, where LTO gets resolved when generating the library > >> but no > >> LTO-symbols are exported in the shared library. This ensure the > >> library > >> has a > >> well defined binary compatible interface and gets LTO optimizations > >> within > >> each library. For some private libraries we use static libraries > >> however, > >> because we don't need binary compatibility, but though we don't need > >> BC > >> > >> between Qt versions, the libraries should still be linkable with > >> different gcc > >> versions (and with different compilers). However when LTO is enabled > >> the > >> static libraries will contain definitions that depend on a single gcc > >> version > >> making it unsuitable for distribution. > >> > >> One solution is to enable fat-lto object files for static libraries > >> but > >> that > >> is both a waste of space and compile time, and disables any LTO > >> optimization > >> within the library. Ideally I would like to have the static library do > >> LTO > >> optimizations internally just like a shared library, but then exported > >> as > >> static library. > >> > >> I suspect this is more of gcc task than ar/ld task, as it basically > >> boils down > >> to gcc doing for a static library what it does for shared library, but > >> maybe > >> export the result as a single combined .o file, that can then be ar'ed > >> into a > >> compatible static library. > >> > >> Is this possible? > > > > Hmm. I think you could partially link the static archive contents into > > a single relocatable object. Or we could add a mode where you do a > > 1to1 LTO link of the objects and stop at the ltrans object files. You > > could stuff those into an archive again. > > > > I'm not sure how far Honza got partial LTO linking to work? > > Parital linking of lto .o files into single non-lto .o file should work > and it will get you cross-module optimization done. The problem is that > without resolution info compiler needs to assume that all symbols > exported by object files are possibly referneced by the later > incremental link and thus the code quality will definitly not be > comparable with what you get for LTO on final binary or DSO. Still > should be better than non-lto build. > I would be curious if it is useful for you in practice. > How would I do that partial link, and what are the requirements? Best regards 'Allan
Re: Resolving LTO symbols for static library boundary
Dne 2018-02-05 18:44, Richard Biener napsal: On February 5, 2018 12:26:58 PM GMT+01:00, Allan Sandfeld Jensenwrote: Hello GCC In trying to make it possible to use LTO for distro-builds of Qt, I have again hit the problem of static libraries. In Qt in general we for LTO rely on a library boundary, where LTO gets resolved when generating the library but no LTO-symbols are exported in the shared library. This ensure the library has a well defined binary compatible interface and gets LTO optimizations within each library. For some private libraries we use static libraries however, because we don't need binary compatibility, but though we don't need BC between Qt versions, the libraries should still be linkable with different gcc versions (and with different compilers). However when LTO is enabled the static libraries will contain definitions that depend on a single gcc version making it unsuitable for distribution. One solution is to enable fat-lto object files for static libraries but that is both a waste of space and compile time, and disables any LTO optimization within the library. Ideally I would like to have the static library do LTO optimizations internally just like a shared library, but then exported as static library. I suspect this is more of gcc task than ar/ld task, as it basically boils down to gcc doing for a static library what it does for shared library, but maybe export the result as a single combined .o file, that can then be ar'ed into a compatible static library. Is this possible? Hmm. I think you could partially link the static archive contents into a single relocatable object. Or we could add a mode where you do a 1to1 LTO link of the objects and stop at the ltrans object files. You could stuff those into an archive again. I'm not sure how far Honza got partial LTO linking to work? Parital linking of lto .o files into single non-lto .o file should work and it will get you cross-module optimization done. The problem is that without resolution info compiler needs to assume that all symbols exported by object files are possibly referneced by the later incremental link and thus the code quality will definitly not be comparable with what you get for LTO on final binary or DSO. Still should be better than non-lto build. I would be curious if it is useful for you in practice. Honza Richard. Best regards 'Allan Jensen
Re: Resolving LTO symbols for static library boundary
On February 5, 2018 12:26:58 PM GMT+01:00, Allan Sandfeld Jensenwrote: >Hello GCC > >In trying to make it possible to use LTO for distro-builds of Qt, I >have again >hit the problem of static libraries. In Qt in general we for LTO rely >on a >library boundary, where LTO gets resolved when generating the library >but no >LTO-symbols are exported in the shared library. This ensure the library >has a >well defined binary compatible interface and gets LTO optimizations >within >each library. For some private libraries we use static libraries >however, >because we don't need binary compatibility, but though we don't need BC > >between Qt versions, the libraries should still be linkable with >different gcc >versions (and with different compilers). However when LTO is enabled >the >static libraries will contain definitions that depend on a single gcc >version >making it unsuitable for distribution. > >One solution is to enable fat-lto object files for static libraries but >that >is both a waste of space and compile time, and disables any LTO >optimization >within the library. Ideally I would like to have the static library do >LTO >optimizations internally just like a shared library, but then exported >as >static library. > >I suspect this is more of gcc task than ar/ld task, as it basically >boils down >to gcc doing for a static library what it does for shared library, but >maybe >export the result as a single combined .o file, that can then be ar'ed >into a >compatible static library. > >Is this possible? Hmm. I think you could partially link the static archive contents into a single relocatable object. Or we could add a mode where you do a 1to1 LTO link of the objects and stop at the ltrans object files. You could stuff those into an archive again. I'm not sure how far Honza got partial LTO linking to work? Richard. >Best regards >'Allan Jensen
Resolving LTO symbols for static library boundary
Hello GCC In trying to make it possible to use LTO for distro-builds of Qt, I have again hit the problem of static libraries. In Qt in general we for LTO rely on a library boundary, where LTO gets resolved when generating the library but no LTO-symbols are exported in the shared library. This ensure the library has a well defined binary compatible interface and gets LTO optimizations within each library. For some private libraries we use static libraries however, because we don't need binary compatibility, but though we don't need BC between Qt versions, the libraries should still be linkable with different gcc versions (and with different compilers). However when LTO is enabled the static libraries will contain definitions that depend on a single gcc version making it unsuitable for distribution. One solution is to enable fat-lto object files for static libraries but that is both a waste of space and compile time, and disables any LTO optimization within the library. Ideally I would like to have the static library do LTO optimizations internally just like a shared library, but then exported as static library. I suspect this is more of gcc task than ar/ld task, as it basically boils down to gcc doing for a static library what it does for shared library, but maybe export the result as a single combined .o file, that can then be ar'ed into a compatible static library. Is this possible? Best regards 'Allan Jensen