Re: [Development] Contribution: include separate libraries with inlines in the Qt installers
I understand. Thank you for participating. On Friday, October 23, 2015 1:19 AM, Thiago Macieira <thiago.macie...@intel.com> wrote: On Thursday 22 October 2015 18:50:22 Dimitar Dobrev wrote: > I don't use GetProcAddress but that's beside the point. Actually, that is entirely the point. And yes, you do. Doesn't matter if you use it directly or not, that's how on Windows you obtain the address of symbols. I'm talking about Windows because the example you gave had Windows- specific flags like LIBS += -lole32. On other systems, that would be dlsym. Unless you've replaced the entire loading of binaries and you have your own ELF/Mach-O/PE-COFF parser. Then you're not using GetProcAddress/dlsym, but have an equivalent that does exactly the same thing. > The point is > that any binding, not just mine, cannot call functions who have not emitted > object code. Correct, if we assume the binding does GetProcAddress/dlsym and uses libffi to place the call. But that's a generalisation. There are bindings that don't do that and simply generate the inlines they need when they compile against the source code. > In other words, the shipped Qt binaries contain a set of > functions that can only be invoked by C++ and nothing else. So I wouldn't > call it a very specialised use case. I still would. Qt is a C++ framework, so anything that is not C++ is a specialised use-case. Note that using the binding isn't the use-case we're talking about here. We're talking about *building* the binding itself, which only a dozen people in the entire world do. > The reason nobody else has asked for > it so far is that each binding deals with it on its own, usually by > generating an additional C layer over Qt which layer has the inlines > compiled in. However, I believe the problem is common to all bindings so a > Qt add-on to contain these symbols would definitely be useful. And I think the way those other bindings have done it is the correct way. Using GetProcAddress/dlsym plus libffi to call C++ functions is too fragile and specialised for us to support. -- Thiago Macieira - thiago.macieira (AT) intel.com Software Architect - Intel Open Source Technology Center ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] Qt binaries with inlined functions
I don't need invisible inlines either but some of them - such as QModelIndex::row() - are visible. Is there any way to emit just the visible inlines? Dimitar On Monday, April 28, 2014 4:12 PM, Thiago Macieira thiago.macie...@intel.com wrote: Em seg 28 abr 2014, às 11:15:23, Koehne Kai escreveu: -Original Message- From: development-bounces+kai.koehne=digia@qt-project.org Subject: [Development] Qt binaries with inlined functions Hello all, I'd like to start a discussion about .https://bugreports.qt- project.org/browse/QTBUG-32995. Please share your thoughts. To get some numbers I tried to add QMAKE_CXXFLAGS += -fkeep-inline-functions `-fkeep-inline-functions' In C, emit `static' functions that are declared `inline' into the object file, even if the function has been inlined into all of its callers. This switch does not affect functions using the `extern inline' extension in GNU C90. In C++, emit any and all inline functions into the object file. DO NOT pass that flag. There's no need to bloat our binaries with functions that cannot be called (remember, we pass -fvisibility-inlines-hidden). -- Thiago Macieira - thiago.macieira (AT) intel.com Software Architect - Intel Open Source Technology Center ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] Qt binaries with inlined functions
Simon, In the description of the issue in JIRA I've pointed out why I don't like the solution with an additional C/C++ wrapper. In short, I would have to compile and pack such a lib for each OS, and then my users would have to deploy that different per OS lib. I'm not sure I understand your answer to my question. Why wouldn't the P/Invoke approach work if I have all necessary symbols? What I'm asking is if there's any way to emit only the exported inlines (for example, by combining -fkeep-inline-functions and -fvisibility-inlines-hidden). This way I would get my symbols and there won't be uncallable functions in the binaries. Dimitar On Tuesday, April 29, 2014 1:03 PM, Simon Hausmann simon.hausm...@digia.com wrote: On Tuesday 29. April 2014 01.30.24 Dimitar Dobrev wrote: I don't need invisible inlines either but some of them - such as QModelIndex::row() - are visible. Is there any way to emit just the visible inlines? I think the pinvoke approach of creating bindings just won't work reliably with C++ APIs that have inline functions then :( But the ABI promise of Qt remains, so can't you create a library yourself that basically contains the inline functions you need and otherwise references the exported symbols from Qt? Your bindings will need to ship that library, but it should continue to work with newer versions of Qt. (It just needs an update when the inline functions change in a newer Qt version) On Monday, April 28, 2014 4:12 PM, Thiago Macieira [...] `-fkeep-inline-functions' In C, emit `static' functions that are declared `inline' into the object file, even if the function has been inlined into all of its callers. This switch does not affect functions using the `extern inline' extension in GNU C90. In C++, emit any and all inline functions into the object file. DO NOT pass that flag. There's no need to bloat our binaries with functions that cannot be called (remember, we pass -fvisibility-inlines-hidden). Ouch yeah, that makes it a no-go :) Simon___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] Qt binaries with inlined functions
They are not emitted. I've opened QtCore with http://www.dependencywalker.com/ and the only symbols I can see for QModelIndex are: _ZNK11QModelIndex4dataEi _ZNK11QModelIndex5childEii _ZNK11QModelIndex5flagsEv _ZNK11QModelIndex6parentEv _ZNK11QModelIndex7isValidEv _ZNK11QModelIndex7siblingEii _ZNK11QModelIndexeqERKS_ _ZNK11QModelIndexltERKS_ _ZNK11QModelIndexneERKS_ Members such as row() or column() are missing. I don't know about other OS-es but on Windows these are all symbols for that class. Dimitar On Tuesday, April 29, 2014 2:17 PM, Allan Sandfeld Jensen k...@carewolf.com wrote: On Tuesday 29 April 2014, Dimitar Dobrev wrote: I don't need invisible inlines either but some of them - such as QModelIndex::row() - are visible. Is there any way to emit just the visible inlines? Shouldn't be needed they are emitted by default. Only static inlines are omitted from the final binary. Note that you need to mark the methods exported due to -fvisibility-inlines-hidden (or remove that flag from your build). Regards `Allan___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] Qt binaries with inlined functions
Allan, I'm not talking about a custom build at all. This whole discussion I started is about how to avoid one. The results you see I've obtained from the Qt 5.2.1 MinGW binaries as downloaded from qt-project.org. On Tuesday, April 29, 2014 2:43 PM, Allan Sandfeld Jensen k...@carewolf.com wrote: On Tuesday 29 April 2014, Dimitar Dobrev wrote: They are not emitted. I've opened QtCore with http://www.dependencywalker.com/ and the only symbols I can see for QModelIndex are: _ZNK11QModelIndex4dataEi _ZNK11QModelIndex5childEii _ZNK11QModelIndex5flagsEv _ZNK11QModelIndex6parentEv _ZNK11QModelIndex7isValidEv _ZNK11QModelIndex7siblingEii _ZNK11QModelIndexeqERKS_ _ZNK11QModelIndexltERKS_ _ZNK11QModelIndexneERKS_ Interesting. Note that all of those methods are also inline, so you have some inlines but not others. Have you tried without -fvisibility-inlines-hidden? (which marks normally visible inline methods invisible unless explicitly declared exported). Regards `Allan___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] Qt binaries with inlined functions
I understand. Thank you all for your time. Kai, I think you can close the issue as Won't Fix. Regards, Dimitar Dobrev On Tuesday, April 29, 2014 6:24 PM, Thiago Macieira thiago.macie...@intel.com wrote: Em ter 29 abr 2014, às 04:45:37, Dimitar Dobrev escreveu: Allan, I'm not talking about a custom build at all. This whole discussion I started is about how to avoid one. The results you see I've obtained from the Qt 5.2.1 MinGW binaries as downloaded from qt-project.org. There won't be a change to the way we build. We're already doing it the right way. -- Thiago Macieira - thiago.macieira (AT) intel.com Software Architect - Intel Open Source Technology Center ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] Qt binaries with inlined functions
I'm not sure I understand your idea about compiling alongside. The binding is C# so I'd have to either use C++/CLI or translate the body of the inline to C# and compile that. The former works only for Windows, the latter is too much work - essentially a C++ to C# converter. What is your suggestion? Sharing with other bindings - now that's a great idea! I hadn't actually thought of the fact that such a wrapper is the same for any binding. If that could become a Qt module as you suggest, and presented as a binary download at qt-project.org, that would be great. Do you know of a tool that can generate such wrappers? The wrapper I used to make with my own code wasn't complete and when I tried to complete it, it proved quite a time-consuming task and I gave up for the time being. I've searched for such tools which I imagine should exist but I was unable to find any. On Tuesday, April 29, 2014 6:55 PM, Milian Wolff milian.wo...@kdab.com wrote: On Monday 28 April 2014 05:37:40 Dimitar Dobrev wrote: snip My problem (and, I'd say, any binding developer's) is that I can't tell users my binding is great, you just need to compile Qt yourselves because the binary downloads don't work well enough - nobody's going to use it. But whoever is going to use your bindings will also have to download some binary for the bindings, right? So why don't you add the missing functions in a wrapper there and compile it alongside your bindings? I.e. what Simon said: But the ABI promise of Qt remains, so can't you create a library yourself that basically contains the inline functions you need and otherwise references the exported symbols from Qt? Your bindings will need to ship that library, but it should continue to work with newer versions of Qt. (It just needs an update when the inline functions change in a newer Qt version) Why does this not work for you? Note that you could even make such a wrapper library official and share it with other bindings. Maybe it could even become an official Qt module! Then eventually consumers of bindings people would just need to download the additional binding-wrapper libraries and everything works as it should? Bye -- Qt Developer Days 2014 - October 6 - 8 at BCC, Berlin Milian Wolff | milian.wo...@kdab.com | Software Engineer KDAB (Deutschland) GmbHCo KG, a KDAB Group company Tel. Germany +49-30-521325470, Sweden (HQ) +46-563-540090 KDAB - Qt Experts - Platform-independent software solutions ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] Qt binaries with inlined functions
I actually use https://github.com/mono/CppSharp . It has a parser and so on but what I was asking for is a tool for generating wrappers. What I've been doing so far is: get each inlined function - get where it's invoked - include the header of the latter in a source file - compile that with -fkeep-inline-functions. Obviously this only works for inlines that happen to be invoked within the same module and is therefore not good. The proper way I am aware of to get all inlines is to generate functions that call them and compile the result. This latter approach turned out more time consuming than I had thought and that's why I've been searching for a tool to generate those functions. Dimitar On Tuesday, April 29, 2014 8:42 PM, Thiago Macieira thiago.macie...@intel.com wrote: Em ter 29 abr 2014, às 09:10:39, Dimitar Dobrev escreveu: Sharing with other bindings - now that's a great idea! I hadn't actually thought of the fact that such a wrapper is the same for any binding. If that could become a Qt module as you suggest, and presented as a binary download at qt-project.org, that would be great. Do you know of a tool that can generate such wrappers? Have you tried Smoke? Or the parser that PySide uses? -- Thiago Macieira - thiago.macieira (AT) intel.com Software Architect - Intel Open Source Technology Center ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] Qt binaries with inlined functions
Thiago, thank you very much for these explanations. About that option, it only works on Windows so it's no good either way. About generating wrappers for inlines, if you know of a tool that can do that - as I mentioned earlier - it would be of great help to me. On Tuesday, April 29, 2014 10:33 PM, Thiago Macieira thiago.macie...@intel.com wrote: Em ter 29 abr 2014, às 08:06:37, Thiago Macieira escreveu: Em ter 29 abr 2014, às 04:45:37, Dimitar Dobrev escreveu: Allan, I'm not talking about a custom build at all. This whole discussion I started is about how to avoid one. The results you see I've obtained from the Qt 5.2.1 MinGW binaries as downloaded from qt-project.org. There won't be a change to the way we build. We're already doing it the right way. Here's also why we can't use -fkeep-inline-functions: $ cat main.cpp #include utility #include algorithm #include vector $ g++ -O3 -S -o - main.cpp .file .ident GCC: (SUSE Linux) 4.8.1 20130909 [gcc-4_8-branch revision 202388] .section .note.GNU-stack,,@progbits $ g++ -O3 -fkeep-inline-functions -S -o - main.cpp | wc -l 2086 This would keep every single, minor and helper inline function from the Standard Library. We can't do that. On Windows, however, we could use the -fkeep-inline-dllexport, which would make GCC match MSVC behaviour, at the cost of bloating the libraries with code that is never called. I'd rather we didn't do it. -- Thiago Macieira - thiago.macieira (AT) intel.com Software Architect - Intel Open Source Technology Center ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] Qt binaries with inlined functions
Unfortunately, I don't. Perhaps you need to add the flag to QMAKE_CFLAGS as well? Dimitar On Monday, April 28, 2014 2:15 PM, Koehne Kai kai.koe...@digia.com wrote: -Original Message- From: development-bounces+kai.koehne=digia@qt-project.org Subject: [Development] Qt binaries with inlined functions Hello all, I'd like to start a discussion about .https://bugreports.qt- project.org/browse/QTBUG-32995. Please share your thoughts. To get some numbers I tried to add QMAKE_CXXFLAGS += -fkeep-inline-functions to mkspecs/common/gcc-base.conf . However, this only results in lots of undefined references when linking qmake . E.g.: g++ -o /home/kakoehne/dev/qt/qt-rls/gcc-4.8-release/qtbase/bin/qmake project.o option.o property.o main.o ioutils.o proitems.o qmakevfs.o qmakeglobals.o qmakeparser.o qmakeevaluator.o qmakebuiltins.o makefile.o unixmake2.o unixmake.o mingw_make.o winmakefile.o projectgenerator.o meta.o makefiledeps.o metamakefile.o xmloutput.o pbuilder_pbx.o msvc_vcproj.o msvc_vcxproj.o msvc_nmake.o msvc_objectmodel.o msbuild_objectmodel.o gbuild.o cesdkhandler.o qtextcodec.o qutfcodec.o qstring.o qstring_compat.o qstringbuilder.o qtextstream.o qiodevice.o qmalloc.o qglobal.o qarraydata.o qbytearray.o qbytearraymatcher.o qdatastream.o qbuffer.o qlist.o qfiledevice.o qfile.o qfilesystementry.o qfilesystemengine.o qfsfileengine.o qfsfileengine_iterator.o qregexp.o qvector.o qbitarray.o qdir.o qdiriterator.o quuid.o qhash.o qfileinfo.o qdatetime.o qstringlist.o qabstractfileengine.o qtemporaryfile.o qmap.o qmetatype.o qsettings.o qsystemerror.o qlibraryinfo.o qvariant.o qvsnprintf.o qlocale.o qlocale_tools.o qlinkedlist.o qnumeric.o qcryptographichash.o qxmlstream.o qxmlutils.o qlogging.o qjson.o qjsondocument.o qjsonparser.o qjsonarray.o qjsonobject.o qjsonvalue.o qfilesystemengine_unix.o qfilesystemiterator_unix.o qfsfileengine_unix.o qlocale_unix.o qtbase/include/QtCore/5.3.0/QtCore/private/../../../../../../../src/qtbase/src/corelib/tools/qsimd_p.h:234: undefined reference to `qt_cpu_features' qtbase/include/QtCore/5.3.0/QtCore/private/../../../../../../../src/qtbase/src/corelib/tools/qsimd_p.h:236: undefined reference to `qDetectCpuFeatures()' qtbase/include/QtCore/5.3.0/QtCore/private/../../../../../../../src/qtbase/src/corelib/tools/qsimd_p.h:237: undefined reference to `qt_cpu_features' qstring.o: In function `qCpuFeatures': qtbase/src/corelib/tools/qsimd_p.h:234: undefined reference to `qt_cpu_features' qtbase/src/corelib/tools/qsimd_p.h:236: undefined reference to `qDetectCpuFeatures()' qtbase/src/corelib/tools/qsimd_p.h:237: undefined reference to `qt_cpu_features' qstring.o: In function `QRegularExpression::operator!=(QRegularExpression const) const': qtbase/src/corelib/tools/qregularexpression.h:128: undefined reference to `QRegularExpression::operator==(QRegularExpression const) const' ... Configure line was configure -release -nomake examples -nomake tests -opensource -confirm-license -prefix `pwd`/qtbase .Do you maybe have a working patch already? Regards Kai ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] Qt binaries with inlined functions
I think it depends on whether you see bindings as a single case or as multiple cases (one per binding). About other, let's say, types of projects - I am not aware. About an option - do you mean an option in the, say, online installer? If it's a compile option, I'm afraid you might as well not bother because it won't work for me. My problem (and, I'd say, any binding developer's) is that I can't tell users my binding is great, you just need to compile Qt yourselves because the binary downloads don't work well enough - nobody's going to use it. Regards, Dimitar On Monday, April 28, 2014 3:29 PM, Koehne Kai kai.koe...@digia.com wrote: -Original Message- From: Olivier Goffart [mailto:oliv...@woboq.com] [...] Maybe it should go in qt_module.prf Thanks, that takes me one step further - linking Qt5Gui fails next. Anyhow, Qt5Core succeeded: Size of libQt5Core.so.5.3.0 without patch:5.9M Size of libQt5Core.so.5.3.0 with patch: 9.3M That's quite a big change! The number of exported symbols (nm -D) for Qt5Core raises from 5440 to 6162. So, just based on this I'd say we shouldn't do this 'just in case'. Would a configure option suffice? Do any other use cases / projects benefit from this? Regards Kai___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
[Development] Qt binaries with inlined functions
Hello all, I'd like to start a discussion about .https://bugreports.qt-project.org/browse/QTBUG-32995.Please share your thoughts. Regards, Dimitar Dobrev___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] Qt binaries with inlined functions
Hi, That's right, MSVC inlines everything by default, and your suggestion is exactly what I need (with the clarification that I also need it for the OS X binaries, which are compiled with Clang and not GCC - I'm not sure about Clang's default behaviour about inlines). About the size, I haven't compiled Qt but the wrapper I used to generate for almost all of the exported inlines in QtCore was about 800 K large so I assume this is approximately how much QtCore would grow. To me, this is little difference and this is why I filed the issue. I'm not sure I understand what you mean about the issue not being specific to qt-project.org binaries. Are you talking about some other binaries? Dimitar On Saturday, April 26, 2014 1:42 PM, Hausmann Simon simon.hausm...@digia.com wrote: Hi, Isn't this also how msvc behaves by default? Then perhaps it would make sense to build with those flags by default with gcc for consistency. I wonder what the overhead in size is. Either way I think this should not be specific to qt-project.org binaries. Simon Fra: Dimitar Dobrev Sendt: 10:44 lørdag 26. april 2014 Til: development@qt-project.org Svar til: Dimitar Dobrev Emne: [Development] Qt binaries with inlined functions Hello all, I'd like to start a discussion about .https://bugreports.qt-project.org/browse/QTBUG-32995.Please share your thoughts. Regards, Dimitar Dobrev___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
