Re: [Qgis-developer] Possible memory leak in rendering code
On 23 April 2013 19:05, Andreas Neumann a.neum...@carto.net wrote: Hi, How comes that the same feature works fine for some but is broken for others? When I have it on (using a combination of Postgis, WMS and GeoTiff) I often have redraw issues and the layer toggeling often fails. Layers are not drawn, but appear to be on with the checkbox in the layer tree or vice versa. I think the main problem is when also using WMS sources. Now that you've mentioned this I'm noticing it all the time. I've ended up switching off render caching and am OK if it gets removed now. Nyall ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [Qgis-developer] Possible memory leak in rendering code
Backbuffering solves a lot of issues with the rendering cache. It was only introduced on Linux so far (to avoid flickering of the canvas) but maybe it should be enabled automatically if rendering cache is enabled. Explanation: Most crashes I had, happened due to recursively starting another rendering loop out of an existing rendering loop. Normally initiated by some user interaction. These user interactions wait in an event queue, which is only processed, a) when the rendering loop is finished or b) when processEvents is called. The backbuffering option removes the call to processEvents. This has some limitations like you are not able to incrementally draw features and you are not able to cancel rendering (Escape). If leaving the render cache in the code, I would at least automatically enable backbuffering as well for the sake of stability. Matthias On Die 23 Apr 2013 04:25:29 CEST, John C. Tull wrote: On Apr 22, 2013, at 2:12 PM, Nyall Dawson nyall.daw...@gmail.com wrote: By the way, it would be a good thing to remove render caching completely before 2.0. In any case it will need to be reworked when moving to multi-threaded rendering... it has only a limited functionality of storing previously rendered map in each layer and reusing it in case the extent has not changed. Unfortunately the rendered image is stored directly within QgsMapLayer class - in order to work properly, the cache should be kept internally within each QgsMapRenderer instance. -1 from me -- please don't do this! I realise that it has some limitations, but render caching makes a huge difference in speed with my workflow. I frequently use layers based off complex PostGIS views which take a long time to generate. With render caching I'm able to selectively toggle layers on/off or rearrange them for comparison without having to wait for QGIS to request a new version of the view from the PostGIS server. If I switch off render caching then every change to layer visibility or ordering triggers a refresh of the view and a painful wait in QGIS. Obviously an ideal solution would be to cache the PostGIS layer locally, but until that's possible render caching helps a lot... Nyall Unfortunately, the render caching feature has never worked on OSX. It is a greyed out option in the settings because the code was never stable on this platform. If it is meant to be a program feature, it would ideally be one that is available on all platforms. Regards, John ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [Qgis-developer] Possible memory leak in rendering code
The pull request got accepted. Thank you Minoru. Alex, could you test if this helps anything for your problem? Matthias On 22.04.2013 12:03, Minoru Akagi wrote: Hi, I found some memory leaks and attached a patch to ticket #6795. Regards, Minoru Akagi 2013/4/22 Matthias Kuhn matthias.k...@gmx.ch: Hi Alex, I found in the past particularly two things making the rendering more stable: * Disable Render Cache * Enable Backbuffer (only availale in X11/Linux) Can you check if one of these two changes in the options fix this issue? (Restart after changing) Cheers, Matthias On Mon 22 Apr 2013 10:50:29 CEST, Alexander Bruy wrote: Hi all, working on QTiles plugin we found one odd issue — when number of canvas redraws is relatively big (around 200 000 - 300 000) QGIS crashed with segmentation fault. Seems there is a memory leak somewhere. I'll try to debug this more, but as I'm not very familiar with rendering code any help and suggestions are welcome. Small test plugin to replicate this issue attached. Also it is possible to use QTiles plugin for this: just set extent and zoom levels so number of tiles is greater that 200 000 and start process. QGIS will crash at 60-70%. ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [Qgis-developer] Possible memory leak in rendering code
Hi, On Tue, Apr 23, 2013 at 8:22 AM, Matthias Kuhn matthias.k...@gmx.ch wrote: This has some limitations like you are not able to incrementally draw features and you are not able to cancel rendering (Escape). I would be very sorry to see this feature go. I use it regularly since I often work with huge datasets. E.g. loading the street network of a whole country from PostGIS, I would have to wait for ages without the possibility to cancel rendering. Best wishes, Anita ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [Qgis-developer] Possible memory leak in rendering code
Hi Anita, Please don't get me wrong. I don't want this to be the default. But it should be the default with rendering cache enabled (if the decision is to keep rendering cache at all). Regards Matthias On Die 23 Apr 2013 10:09:24 CEST, Anita Graser wrote: Hi, On Tue, Apr 23, 2013 at 8:22 AM, Matthias Kuhn matthias.k...@gmx.ch mailto:matthias.k...@gmx.ch wrote: This has some limitations like you are not able to incrementally draw features and you are not able to cancel rendering (Escape). I would be very sorry to see this feature go. I use it regularly since I often work with huge datasets. E.g. loading the street network of a whole country from PostGIS, I would have to wait for ages without the possibility to cancel rendering. Best wishes, Anita ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [Qgis-developer] Possible memory leak in rendering code
I am surprised that people really used the render caching. I had all sorts of issues so I did not use it. To me it was really broken. I think we should remove it and reintroduce it when multithreading is implemented and the feature works as expected. Better not ship it than ship in a broken state. Anita Graser anitagra...@gmx.at schrieb: Hi, On Tue, Apr 23, 2013 at 8:22 AM, Matthias Kuhn matthias.k...@gmx.ch wrote: This has some limitations like you are not able to incrementally draw features and you are not able to cancel rendering (Escape). I would be very sorry to see this feature go. I use it regularly since I often work with huge datasets. E.g. loading the street network of a whole country from PostGIS, I would have to wait for ages without the possibility to cancel rendering. Best wishes, Anita ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer -- Diese Nachricht wurde von meinem Android-Mobiltelefon mit K-9 Mail gesendet.___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [Qgis-developer] Possible memory leak in rendering code
I'm the opposite. I have it enabled all the time and don't really have any issues with it. It's not doing any harm leaving it in IMO so -1 to remove it. - Nathan On Tue, Apr 23, 2013 at 6:38 PM, Andreas Neumann a.neum...@carto.netwrote: I am surprised that people really used the render caching. I had all sorts of issues so I did not use it. To me it was really broken. I think we should remove it and reintroduce it when multithreading is implemented and the feature works as expected. Better not ship it than ship in a broken state. Anita Graser anitagra...@gmx.at schrieb: Hi, On Tue, Apr 23, 2013 at 8:22 AM, Matthias Kuhn matthias.k...@gmx.chwrote: This has some limitations like you are not able to incrementally draw features and you are not able to cancel rendering (Escape). I would be very sorry to see this feature go. I use it regularly since I often work with huge datasets. E.g. loading the street network of a whole country from PostGIS, I would have to wait for ages without the possibility to cancel rendering. Best wishes, Anita -- Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer -- Diese Nachricht wurde von meinem Android-Mobiltelefon mit K-9 Mail gesendet. ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [Qgis-developer] Possible memory leak in rendering code
Hi, How comes that the same feature works fine for some but is broken for others? When I have it on (using a combination of Postgis, WMS and GeoTiff) I often have redraw issues and the layer toggeling often fails. Layers are not drawn, but appear to be on with the checkbox in the layer tree or vice versa. I think the main problem is when also using WMS sources. Really strange ... Andreas Am 23.04.2013 10:49, schrieb Nathan Woodrow: I'm the opposite. I have it enabled all the time and don't really have any issues with it. It's not doing any harm leaving it in IMO so -1 to remove it. - Nathan On Tue, Apr 23, 2013 at 6:38 PM, Andreas Neumann a.neum...@carto.netwrote: I am surprised that people really used the render caching. I had all sorts of issues so I did not use it. To me it was really broken. I think we should remove it and reintroduce it when multithreading is implemented and the feature works as expected. Better not ship it than ship in a broken state. Anita Graser anitagra...@gmx.at schrieb: Hi, On Tue, Apr 23, 2013 at 8:22 AM, Matthias Kuhn matthias.k...@gmx.chwrote: This has some limitations like you are not able to incrementally draw features and you are not able to cancel rendering (Escape). I would be very sorry to see this feature go. I use it regularly since I often work with huge datasets. E.g. loading the street network of a whole country from PostGIS, I would have to wait for ages without the possibility to cancel rendering. Best wishes, Anita -- Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer -- Diese Nachricht wurde von meinem Android-Mobiltelefon mit K-9 Mail gesendet. ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [Qgis-developer] Possible memory leak in rendering code
Hi, this changes doesn't solve issue for me. Crash happens even if only vector layers with or without labels used. Just finished testing with disabled caching and enabled backbuffer (only vector layers with Single symbol style, one layer labeled using PAL and scale-dependent labeling is enabled). Same result — first rendering becomes very slow, QGIS eats memory and finally crashed. Here is last console messages for this case: Debug: src/core/qgscoordinatetransform.cpp: 535: (transformBoundingBox) Projected extent: 82.7435302734374858,54.9302946986374678 : 82.749023437500,54.9334578175455306 Debug: src/core/qgsvectorlayer.cpp: 301: (drawLabels) Starting draw of labels: road_l_osm20130415140006374 Debug: src/core/qgspallabeling.cpp: 1790: (drawLabeling) LABELING work: 17 ms ... labels# 0 Debug: src/core/qgspallabeling.cpp: 1977: (drawLabeling) LABELING draw: 2 ms Debug: src/core/qgsmaprenderer.cpp: 689: (render) Rendering completed in (seconds): 92.944 terminate called after throwing an instance of 'std::bad_alloc' what(): std::bad_alloc On Tue, 23 Apr 2013 08:24:04 +0200 Matthias Kuhn matthias.k...@gmx.ch wrote: The pull request got accepted. Thank you Minoru. Alex, could you test if this helps anything for your problem? Matthias -- Alexander Bruy ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [Qgis-developer] Possible memory leak in rendering code
On 04/23/2013 10:38 AM, Andreas Neumann wrote: I am surprised that people really used the render caching. I had all sorts of issues so I did not use it. To me it was really broken. I think we should remove it and reintroduce it when multithreading is implemented and the feature works as expected. Better not ship it than ship in a broken state. I have the same experience. -- Ivan Mincik ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [Qgis-developer] Possible memory leak in rendering code
Obviously an ideal solution would be to cache the PostGIS layer locally, but until that's possible render caching helps a lot... Also you can use 'materialized views' [1] in Your database. 1 - http://tech.jonathangardner.net/wiki/PostgreSQL/Materialized_Views -- Ivan Mincik ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [Qgis-developer] Possible memory leak in rendering code
On 04/23/2013 11:05 AM, Andreas Neumann wrote: Hi, How comes that the same feature works fine for some but is broken for others? When I have it on (using a combination of Postgis, WMS and GeoTiff) I often have redraw issues and the layer toggeling often fails. Layers are not drawn, but appear to be on with the checkbox in the layer tree or vice versa. I think the main problem is when also using WMS sources. Yes, I see the same layer redrawing problems when using WMS and PostGIS layers. -- Ivan Mincik ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [Qgis-developer] Possible memory leak in rendering code
Hi, I found some memory leaks and attached a patch to ticket #6795. Regards, Minoru Akagi 2013/4/22 Matthias Kuhn matthias.k...@gmx.ch: Hi Alex, I found in the past particularly two things making the rendering more stable: * Disable Render Cache * Enable Backbuffer (only availale in X11/Linux) Can you check if one of these two changes in the options fix this issue? (Restart after changing) Cheers, Matthias On Mon 22 Apr 2013 10:50:29 CEST, Alexander Bruy wrote: Hi all, working on QTiles plugin we found one odd issue — when number of canvas redraws is relatively big (around 200 000 - 300 000) QGIS crashed with segmentation fault. Seems there is a memory leak somewhere. I'll try to debug this more, but as I'm not very familiar with rendering code any help and suggestions are welcome. Small test plugin to replicate this issue attached. Also it is possible to use QTiles plugin for this: just set extent and zoom levels so number of tiles is greater that 200 000 and start process. QGIS will crash at 60-70%. ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [Qgis-developer] Possible memory leak in rendering code
Hi Matthias, thanks for hints. Just tested with disabled Renderer Cache and again get crash. Here is last console messages: Warning: QPainter::end: Painter ended with 2 saved states Debug: src/core/qgsmessagelog.cpp: 45: (logMessage) 2013-04-22T15:19:22 Qt[1] QPainter::end: Painter ended with 2 saved states Warning: QNativeImage: Unable to attach to shared memory segment. Segmentation fault On Mon, 22 Apr 2013 10:54:04 +0200 Matthias Kuhn matthias.k...@gmx.ch wrote: Hi Alex, I found in the past particularly two things making the rendering more stable: * Disable Render Cache * Enable Backbuffer (only availale in X11/Linux) Can you check if one of these two changes in the options fix this issue? (Restart after changing) Cheers, Matthias -- Alexander Bruy ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [Qgis-developer] Possible memory leak in rendering code
On Mon, Apr 22, 2013 at 10:54 AM, Matthias Kuhn matthias.k...@gmx.ch wrote: Hi Alex, I found in the past particularly two things making the rendering more stable: * Disable Render Cache * Enable Backbuffer (only availale in X11/Linux) By the way, it would be a good thing to remove render caching completely before 2.0. In any case it will need to be reworked when moving to multi-threaded rendering... it has only a limited functionality of storing previously rendered map in each layer and reusing it in case the extent has not changed. Unfortunately the rendered image is stored directly within QgsMapLayer class - in order to work properly, the cache should be kept internally within each QgsMapRenderer instance. Martin ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [Qgis-developer] Possible memory leak in rendering code
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Il 22/04/2013 13:42, Martin Dobias ha scritto: By the way, it would be a good thing to remove render caching completely before 2.0. In any case it will need to be reworked when Hi Martin, it makes sense. Any objections in removing it? All the best. - -- Paolo Cavallini - Faunalia www.faunalia.eu Full contact details at www.faunalia.eu/pc Nuovi corsi QGIS e PostGIS: http://www.faunalia.it/calendario -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.12 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAlF1VTMACgkQ/NedwLUzIr7uAwCeO33pD5gF/efeFbU7vLNXJ4+w RSYAmgK1LB1LAUEoYQiYLxUixmetXzBJ =TTRv -END PGP SIGNATURE- ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [Qgis-developer] Possible memory leak in rendering code
On Apr 22, 2013, at 2:12 PM, Nyall Dawson nyall.daw...@gmail.com wrote: By the way, it would be a good thing to remove render caching completely before 2.0. In any case it will need to be reworked when moving to multi-threaded rendering... it has only a limited functionality of storing previously rendered map in each layer and reusing it in case the extent has not changed. Unfortunately the rendered image is stored directly within QgsMapLayer class - in order to work properly, the cache should be kept internally within each QgsMapRenderer instance. -1 from me -- please don't do this! I realise that it has some limitations, but render caching makes a huge difference in speed with my workflow. I frequently use layers based off complex PostGIS views which take a long time to generate. With render caching I'm able to selectively toggle layers on/off or rearrange them for comparison without having to wait for QGIS to request a new version of the view from the PostGIS server. If I switch off render caching then every change to layer visibility or ordering triggers a refresh of the view and a painful wait in QGIS. Obviously an ideal solution would be to cache the PostGIS layer locally, but until that's possible render caching helps a lot... Nyall Unfortunately, the render caching feature has never worked on OSX. It is a greyed out option in the settings because the code was never stable on this platform. If it is meant to be a program feature, it would ideally be one that is available on all platforms. Regards, John ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer
