Hi all, > Am I the only QGIS dev highly frustrated by the bottleneck of > sip-build when building QGIS?
No, you're not alone. I also experience crashes or freezes when building and I have considerably reduced the number of processes to be used to avoid these... > Can we do something (funding upstream?) > to improve that? We can still reach out Phil Thompson on the Python-SIP issue tracker [0] I have done it several times and things have been improved or fixed (unique_ptr management for instance lately). Problem is that we depend on SIP versions shipped in linux distributions, so we would have to wait quite a long time before using those improvments. > - we bind to Python a very large part of our header files (more than > half). I've always found that the amount of things exposed to Python > was excessive. IMHO we should be much more conservative, and wait > for plugin authors to ask for an API to be exposed, rather than > expose it prematurely. > I completely agree. And I'm wondering what part of the API is really used by the existing plugins, I'm suspecting that it's a very small part. A while ago, I take a look about what Blender was exposing in its Python API. It's so small in comparison. If other contributors agree with this statement, maybe we could try to work on a Policy to avoid adding any thing new to the Python API that it's not worth it. Regards, Julien [0]: https://github.com/Python-SIP/sip > Hi, > > sorry for the eye-catching email title, couldn't resist. > > Am I the only QGIS dev highly frustrated by the bottleneck of > sip-build when building QGIS? Can we do something (funding upstream?) > to improve that? > > Also some mind bogging stats: > > - size of all QGIS headers: find ../src -name "*.h" -exec cat {} \; | > wc -l : 570 288 > - size of QGIS .cpp + .ui: find ../src \( -name "*.cpp" -o -name > "*.ui" \) -exec cat {} \; | wc -l : 1 497 216 > > So QGIS is just a 2 million LoC (excluding tests) code base > > - size of .sip generated files: find . -name "*.sip" -exec cat {} \; | > wc -l : 335 425 > - size of SIP generated .cpp files: find python/ -name "*.cpp" -exec > cat {} \; | wc -l: 7 532 645 . 7.5 millions LoC !!! > - size of MOC generated .cpp files: find src/ -name "*.cpp" -exec cat > {} \; | wc -l : 2 051 769 > > So 2 observations: > > - we bind to Python a very large part of our header files (more than > half). I've always found that the amount of things exposed to Python > was excessive. IMHO we should be much more conservative, and wait > for plugin authors to ask for an API to be exposed, rather than > expose it prematurely. > > - SIP generates > 20 times more lines of .cpp than .sip files!!! Or > 3.75 times more lines of code than QGIS actual logic. > > That shows in (stripped) .so files: > > - output/python/qgis/*.so : 130 MB > - output/lib/*.so.4.1.0 : 170 MB. > > So not as crazy as the 7.5 vs 2 MLOC of ratio, but still. > > Comparison with GDAL and SWIG: > - size of public .h headers: 50 162 . > - size of Python SWIG source .i files: ~43 000 (hard to count > exactly as there's some bits of Java and C# in this count) > - size of SWIG Python generated .cpp files: 164 151 . > So a 4x only ratio of .cpp vs .i > > I don't recommend SWIG, but something looks fishy in the kingdom of SIP > > Even -- Julien Cabieces Senior Developer at Oslandia [email protected] _______________________________________________ QGIS-Developer mailing list [email protected] List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
