Re: [QGIS-Developer] SIP is insane
For posterity again: while below works fine at compilation time, it breaks at runtime due to Qt / SIP version mismatches :-( Thanks for the hint. That does improve things (sip-build time) indeed. For posterity, I managed to use it without altering too much my build env with: python3 -m venv sip_venv source sip_venv/bin/activate pip install sip PyQt-builder cmake -USIP_BUILD_EXECUTABLE .. -- http://www.spatialys.com My software is free, but my time generally not. ___ QGIS-Developer mailing list [email protected] List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [QGIS-Developer] SIP is insane
Nice! With sip code being python, I had just patched my system sip with David's diff, until 6.15 was included in Debian testing. Best, Stefanos On Tue, Jun 9, 2026, 4:46 PM Even Rouault wrote: > > Le 09/06/2026 à 15:19, Stefanos Natsis a écrit : > > Hi all, > > > Am I the only QGIS dev highly frustrated by the bottleneck of sip-build >> when building QGIS? Can we do something (funding upstream?) to improve >> that? > > > I used to be quite frustrated when sip4 support was dropped, as sip-build > times were really terrible with sip 6 at the time. > After David's fixes in Oct/Nov 2025 though, things are back to normal. > With sip-tools v6.15+ I don't think sip-build is a bottleneck any more for > me. > > Thanks for the hint. That does improve things (sip-build time) indeed. For > posterity, I managed to use it without altering too much my build env with: > > python3 -m venv sip_venv > source sip_venv/bin/activate > pip install sip PyQt-builder > cmake -USIP_BUILD_EXECUTABLE .. > > -- http://www.spatialys.com > My software is free, but my time generally not. > > ___ QGIS-Developer mailing list [email protected] List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [QGIS-Developer] SIP is insane
Le 09/06/2026 à 15:19, Stefanos Natsis a écrit : Hi all, Am I the only QGIS dev highly frustrated by the bottleneck of sip-build when building QGIS? Can we do something (funding upstream?) to improve that? I used to be quite frustrated when sip4 support was dropped, as sip-build times were really terrible with sip 6 at the time. After David's fixes in Oct/Nov 2025 though, things are back to normal. With sip-tools v6.15+ I don't think sip-build is a bottleneck any more for me. Thanks for the hint. That does improve things (sip-build time) indeed. For posterity, I managed to use it without altering too much my build env with: python3 -m venv sip_venv source sip_venv/bin/activate pip install sip PyQt-builder cmake -USIP_BUILD_EXECUTABLE .. -- http://www.spatialys.com My software is free, but my time generally not. ___ QGIS-Developer mailing list [email protected] List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [QGIS-Developer] SIP is insane
Hi all, Am I the only QGIS dev highly frustrated by the bottleneck of sip-build > when building QGIS? Can we do something (funding upstream?) to improve > that? I used to be quite frustrated when sip4 support was dropped, as sip-build times were really terrible with sip 6 at the time. After David's fixes in Oct/Nov 2025 though, things are back to normal. With sip-tools v6.15+ I don't think sip-build is a bottleneck any more for me. I'm more annoyed that touching any enum or docstring in qgis.h will trigger an almost full rebuild of the whole project! Though, I still often do `ninja qgis` during iterative rebuilds to avoid sip-build if not strictly necessary. > - we bind to Python a very large part of our header files (more than > half). I've always found that the amount of things exposed to Python was > excessive. IMHO we should be much more conservative, and wait for plugin > authors to ask for an API to be exposed, rather than expose it prematurely. > My personal preference would be to expose as much as the stable API promise allows us to, then encourage development of new and often niche features to be done as python plugins instead of having them in core. Best, Stefanos ___ QGIS-Developer mailing list [email protected] List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [QGIS-Developer] SIP is insane
Hi Regis, > On 08/06/2026 10:00, Julien Cabieces via QGIS-Developer wrote: >> I completely agree. And I'm wondering what part of the API is really >> used by the existing plugins, I'm suspecting that it's a very small part. > > Hi, I'm asking as a non-dev, could we imagine scanning all the public > repository for the classes and method that are actually used by > plugins? > That could help having a real picture here. I'd be +1 for a grant > funding here > I also wanted to something alike, just as a matter of curiosity. I'm wondering for instance if plugins are really using the 200 widgets we are exposing. Some widget absolutely make sense (QgsFileWidget, QgsScaleWidget...) but I have more doubt for some (the properties widget for instance). But I rarely develop plugin so maybe I'm missing the big picture here. Regards, Julien > > Cheers > > Régis > > > ___ > QGIS-Developer mailing list > [email protected] > List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer > Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer -- Julien Cabieces Senior Developer at Oslandia [email protected] ___ QGIS-Developer mailing list [email protected] List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [QGIS-Developer] SIP is insane
Hi all, Regarding switching to "Qt for Python", I made a report on where I was last time I worked on a possible transition to Qt for Python (december 2023) [0] My feelings: - While trying to transition from SIP to QtForPython, I met so many surprises/special-cases/SIP-specific-behaviors that I think we are now very tightly bounded to it - It's very difficult to predict the needed time required to make the whole transition, you have no idea how many surprises you gonna met - Even if I enjoyed many things from QtForPython, we have no garantee that it would be better in the end We could try to lessen adherence to SIP to ease a possible future transition (replace SIP_FACTORY/SIP_TRANSFER with unique_ptr for instance). > Do we have any advantage migrating to PySide (now it is called the official > Qt for Python)? > If yes I am more than willing to adjust my plug-ins. Normally, no adjustement (or very few) would be needed (for thoses importing from qgis.PyQt at least) Regards, Julien [0] https://github.com/qgis/QGIS-Enhancement-Proposals/issues/237#issuecomment-1839008093 > Hi all, > > Just want to remind you about the QEP about switching to PySide (Qt > for Python) [0]. > But I have no idea if such a move will solve issues raised here. > > [0] https://github.com/qgis/QGIS-Enhancement-Proposals/issues/237 -- Julien Cabieces Senior Developer at Oslandia [email protected] ___ QGIS-Developer mailing list [email protected] List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [QGIS-Developer] SIP is insane
Hi all, I just wanted to ask the same as Alex already did. Do we have any advantage migrating to PySide (now it is called the official Qt for Python)? If yes I am more than willing to adjust my plug-ins. Regards Werner signature.asc Description: PGP signature ___ QGIS-Developer mailing list [email protected] List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [QGIS-Developer] SIP is insane
Hi all, Just want to remind you about the QEP about switching to PySide (Qt for Python) [0]. But I have no idea if such a move will solve issues raised here. [0] https://github.com/qgis/QGIS-Enhancement-Proposals/issues/237 -- Alexander Bruy ___ QGIS-Developer mailing list [email protected] List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [QGIS-Developer] SIP is insane
On 6/6/26 02:18, Even Rouault via QGIS-Developer wrote: I've always found that the amount of things exposed to Python was excessive. IMHO we should be much more conservative, and wait for plugin authors to ask for an API to be exposed, rather than expose it prematurely. Not sure which part this would be, but it reminds me of the "old days" when Python bindings were made by hand and a big part of the API was not exposed. As a plugin developer I needed to ask core developers to expose a class or function to Python and wait for it to happen. Then I needed to build QGIS (which I can) but after finishing the plugin my customers had to wait for months (if not a years) for the next QGIS release to be able to run the plugin on their company wide QGIS (LTR) installation. Not nice for the core developers (I guess making Python bindings is not the most fun task), not nice for plugin developers and not nice for QGIS users in need for a new plugin. I'm also wondering how many plugins would break if we would suddenly quit exposing part of the API. Even if we scan the entire plugin repository, there are many more plugins in the world. So, I'm really happy with the current approach of exposing everything. If this can be more efficient somehow that would of course be wonderful. Kind regards, Raymond ___ QGIS-Developer mailing list [email protected] List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [QGIS-Developer] SIP is insane
On 08/06/2026 10:00, Julien Cabieces via QGIS-Developer wrote: I completely agree. And I'm wondering what part of the API is really used by the existing plugins, I'm suspecting that it's a very small part. Hi, I'm asking as a non-dev, could we imagine scanning all the public repository for the classes and method that are actually used by plugins? That could help having a real picture here. I'd be +1 for a grant funding here Cheers Régis ___ QGIS-Developer mailing list [email protected] List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [QGIS-Developer] SIP is insane
Hi all,
> Am I the only QGIS dev highly frustrated by the bottleneck of
> sip-build when building QGIS?
No, you're not alone. I also experience crashes or freezes when building
and I have considerably reduced the number of processes to be used to
avoid these...
> Can we do something (funding upstream?)
> to improve that?
We can still reach out Phil Thompson on the Python-SIP issue tracker [0]
I have done it several times and things have been improved or fixed
(unique_ptr management for instance lately).
Problem is that we depend on SIP versions shipped in linux distributions,
so we would have to wait quite a long time before using those improvments.
> - we bind to Python a very large part of our header files (more than
> half). I've always found that the amount of things exposed to Python
> was excessive. IMHO we should be much more conservative, and wait
> for plugin authors to ask for an API to be exposed, rather than
> expose it prematurely.
>
I completely agree. And I'm wondering what part of the API is really
used by the existing plugins, I'm suspecting that it's a very small part.
A while ago, I take a look about what Blender was exposing in its Python
API. It's so small in comparison.
If other contributors agree with this statement, maybe we could try to
work on a Policy to avoid adding any thing new to the Python API that
it's not worth it.
Regards,
Julien
[0]: https://github.com/Python-SIP/sip
> Hi,
>
> sorry for the eye-catching email title, couldn't resist.
>
> Am I the only QGIS dev highly frustrated by the bottleneck of
> sip-build when building QGIS? Can we do something (funding upstream?)
> to improve that?
>
> Also some mind bogging stats:
>
> - size of all QGIS headers: find ../src -name "*.h" -exec cat {} \; |
> wc -l : 570 288
> - size of QGIS .cpp + .ui: find ../src \( -name "*.cpp" -o -name
> "*.ui" \) -exec cat {} \; | wc -l : 1 497 216
>
> So QGIS is just a 2 million LoC (excluding tests) code base
>
> - size of .sip generated files: find . -name "*.sip" -exec cat {} \; |
> wc -l : 335 425
> - size of SIP generated .cpp files: find python/ -name "*.cpp" -exec
> cat {} \; | wc -l: 7 532 645 . 7.5 millions LoC !!!
> - size of MOC generated .cpp files: find src/ -name "*.cpp" -exec cat
> {} \; | wc -l : 2 051 769
>
> So 2 observations:
>
> - we bind to Python a very large part of our header files (more than
> half). I've always found that the amount of things exposed to Python
> was excessive. IMHO we should be much more conservative, and wait
> for plugin authors to ask for an API to be exposed, rather than
> expose it prematurely.
>
> - SIP generates > 20 times more lines of .cpp than .sip files!!! Or
> 3.75 times more lines of code than QGIS actual logic.
>
> That shows in (stripped) .so files:
>
> - output/python/qgis/*.so : 130 MB
> - output/lib/*.so.4.1.0 : 170 MB.
>
> So not as crazy as the 7.5 vs 2 MLOC of ratio, but still.
>
> Comparison with GDAL and SWIG:
> - size of public .h headers: 50 162 .
> - size of Python SWIG source .i files: ~43 000 (hard to count
> exactly as there's some bits of Java and C# in this count)
> - size of SWIG Python generated .cpp files: 164 151 .
> So a 4x only ratio of .cpp vs .i
>
> I don't recommend SWIG, but something looks fishy in the kingdom of SIP
>
> Even
--
Julien Cabieces
Senior Developer at Oslandia
[email protected]
___
QGIS-Developer mailing list
[email protected]
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [QGIS-Developer] SIP is insane
Hi Even,
I hear your scream and I scream with you :)
Something has definitely worsened after the shift to QT6 (even if I'm
not sure if the QT version has anything to do).
I've come to the point where I cannot build QGIS on my laptop (16GB
RAM 4 physical cores) anymore and even on my office PC (32 GB RAM and
8 physical cores) it occasionally crashes for OOM or freezes.
All the times that it happened, sip-build was eating up all resources.
I am sorry this is just a "me too" and I have no solution for this issue.
On Sat, Jun 6, 2026 at 3:16 AM Even Rouault via QGIS-Developer
wrote:
>
> Hi,
>
> sorry for the eye-catching email title, couldn't resist.
>
> Am I the only QGIS dev highly frustrated by the bottleneck of sip-build
> when building QGIS? Can we do something (funding upstream?) to improve that?
>
> Also some mind bogging stats:
>
> - size of all QGIS headers: find ../src -name "*.h" -exec cat {} \; |
> wc -l : 570 288
> - size of QGIS .cpp + .ui: find ../src \( -name "*.cpp" -o -name "*.ui"
> \) -exec cat {} \; | wc -l : 1 497 216
>
> So QGIS is just a 2 million LoC (excluding tests) code base
>
> - size of .sip generated files: find . -name "*.sip" -exec cat {} \; |
> wc -l : 335 425
> - size of SIP generated .cpp files: find python/ -name "*.cpp" -exec
> cat {} \; | wc -l: 7 532 645 . 7.5 millions LoC !!!
> - size of MOC generated .cpp files: find src/ -name "*.cpp" -exec cat {}
> \; | wc -l : 2 051 769
>
> So 2 observations:
>
> - we bind to Python a very large part of our header files (more than
> half). I've always found that the amount of things exposed to Python was
> excessive. IMHO we should be much more conservative, and wait for plugin
> authors to ask for an API to be exposed, rather than expose it prematurely.
>
> - SIP generates > 20 times more lines of .cpp than .sip files!!! Or
> 3.75 times more lines of code than QGIS actual logic.
>
> That shows in (stripped) .so files:
>
> - output/python/qgis/*.so : 130 MB
> - output/lib/*.so.4.1.0 : 170 MB.
>
> So not as crazy as the 7.5 vs 2 MLOC of ratio, but still.
>
> Comparison with GDAL and SWIG:
> - size of public .h headers: 50 162 .
> - size of Python SWIG source .i files: ~43 000 (hard to count exactly
> as there's some bits of Java and C# in this count)
> - size of SWIG Python generated .cpp files: 164 151 .
> So a 4x only ratio of .cpp vs .i
>
> I don't recommend SWIG, but something looks fishy in the kingdom of SIP
>
> Even
>
> --
> http://www.spatialys.com
> My software is free, but my time generally not.
>
> ___
> QGIS-Developer mailing list
> [email protected]
> List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
> Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
--
Alessandro Pasotti
QCooperative: www.qcooperative.net
ItOpen: www.itopen.it
___
QGIS-Developer mailing list
[email protected]
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [QGIS-Developer] SIP is insane
Hi,
as you might remember, I did some work on improving SIP compilation
times last year. I managed to improve performance, but sadly not fully
refactor SIP to allow for a per-class or per-header build. I'll try to
dig out some of my notes and memories on why it was hard if you want.
That said, I only looked at improving performance of sip-build, maybe
there is some low-hanging fruit in the generated .cpp files.
SIP is developed pretty much solely by Phil Thompson, he has a company
called Riverbank Computing. No idea how interested he'd be in being
contracted for a large refactor, though.
David Koňařík
On Sat, 2026-06-06 at 02:18 +0200, Even Rouault via QGIS-Developer
wrote:
> Hi,
>
> sorry for the eye-catching email title, couldn't resist.
>
> Am I the only QGIS dev highly frustrated by the bottleneck of sip-
> build
> when building QGIS? Can we do something (funding upstream?) to
> improve that?
>
> Also some mind bogging stats:
>
> - size of all QGIS headers: find ../src -name "*.h" -exec cat {} \;
> |
> wc -l : 570 288
> - size of QGIS .cpp + .ui: find ../src \( -name "*.cpp" -o -name
> "*.ui"
> \) -exec cat {} \; | wc -l : 1 497 216
>
> So QGIS is just a 2 million LoC (excluding tests) code base
>
> - size of .sip generated files: find . -name "*.sip" -exec cat {} \;
> |
> wc -l : 335 425
> - size of SIP generated .cpp files: find python/ -name "*.cpp" -
> exec
> cat {} \; | wc -l: 7 532 645 . 7.5 millions LoC !!!
> - size of MOC generated .cpp files: find src/ -name "*.cpp" -exec cat
> {}
> \; | wc -l : 2 051 769
>
> So 2 observations:
>
> - we bind to Python a very large part of our header files (more than
> half). I've always found that the amount of things exposed to Python
> was
> excessive. IMHO we should be much more conservative, and wait for
> plugin
> authors to ask for an API to be exposed, rather than expose it
> prematurely.
>
> - SIP generates > 20 times more lines of .cpp than .sip files!!! Or
> 3.75 times more lines of code than QGIS actual logic.
>
> That shows in (stripped) .so files:
>
> - output/python/qgis/*.so : 130 MB
> - output/lib/*.so.4.1.0 : 170 MB.
>
> So not as crazy as the 7.5 vs 2 MLOC of ratio, but still.
>
> Comparison with GDAL and SWIG:
> - size of public .h headers: 50 162 .
> - size of Python SWIG source .i files: ~43 000 (hard to count
> exactly
> as there's some bits of Java and C# in this count)
> - size of SWIG Python generated .cpp files: 164 151 .
> So a 4x only ratio of .cpp vs .i
>
> I don't recommend SWIG, but something looks fishy in the kingdom of
> SIP
>
> Even
___
QGIS-Developer mailing list
[email protected]
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
