Re: [Qgis-developer] min/max for rasters
AFAIK The strategy of 2-98 is usually useful when there a "noised image", because we assume that the noise is a white noise and it is randomized an isolated spikes. THis is absolutely a right theory and really useful, ma what kind of imagge are usually used in a gis system. If we think at the ortophoto image the noise could be really happened because they came from a photo-sensor. But is we think to a artificial image, like the 2-colors balck-white images named "carta tecnica" thata are trasposition of vectorial data. Them are no noised images and has a really thin lines. Also the artifical thematic chars with colors and point and symbols and lines (outline and so on) are noise-less images. Don't forget to think also to geological charts. Are all noise-less images. So what kind of image are more used in a GIS system ? This is not simple question. The response is , "it is dependent by the kind of work you should do.". But also another question is: Usually the ortophoto are not simple to have . They are produced and have a license. The thematic images are more easy to produce and are often without a license or has a free license. More often the ortophoto images are available from a WMS system, and this is a solution that deny the use of the 2-98 strategy. Andrea. On 14/12/2013 07:17, Paolo Cavallini wrote: Il 13/12/2013 20:18, Radim Blazek ha scritto: Can you describe some examples where 2-98% is a problem (data type, number of bands, map content, features/phenomena represented by those 2+2%,...) so that we can think about it better? Example #1 (less problematic): dtm and their legend are always shown wrong; newbies do not understand why Example #2 (more serious): rasterizing sparse vectors (e.g. rivers) results in a black rectangle, as the number of pixels with valid data is <2%. In fact, I think we should help users more, e.g. by applying non linear colour scaling (log, exp) in case of very skewed raster values distribution: if data are more or less normally distributed, no cut is applied, and linear scaling is used; if they are badly skewdw or with outliers, apply a non linear colour scaling. With some thinking, this should solve most if not all user cases, without asking a normal user to understand much about raster stats. However, in my case the general setting "use min/max" does not seem to be working. Thanks for your thoughts. ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [Qgis-developer] min/max for rasters
Il 13/12/2013 20:18, Radim Blazek ha scritto: >> Can you describe some examples where 2-98% is a problem (data type, >> number of bands, map content, features/phenomena represented by those >> 2+2%,...) so that we can think about it better? Example #1 (less problematic): dtm and their legend are always shown wrong; newbies do not understand why Example #2 (more serious): rasterizing sparse vectors (e.g. rivers) results in a black rectangle, as the number of pixels with valid data is <2%. In fact, I think we should help users more, e.g. by applying non linear colour scaling (log, exp) in case of very skewed raster values distribution: if data are more or less normally distributed, no cut is applied, and linear scaling is used; if they are badly skewdw or with outliers, apply a non linear colour scaling. With some thinking, this should solve most if not all user cases, without asking a normal user to understand much about raster stats. However, in my case the general setting "use min/max" does not seem to be working. Thanks for your thoughts. -- Paolo Cavallini - www.faunalia.eu QGIS & PostGIS courses: http://www.faunalia.eu/training.html ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [Qgis-developer] min/max for rasters
On Fri, Dec 13, 2013 at 9:55 AM, Paolo Cavallini wrote: > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA1 > > Hi all. > The default for raster rendering is to cut values at 2-98%. This is > inappropriate in many cases, and confusing for users. Is that OK if I > change the default to min/max? Min/max is inappropriate in many other cases. It is quite common to get data with few outliers. The result in such cases with min/max is black rectangle which is very frustrating and that was the case with 1.8. I believe that it is better to get inappropriately rendered picture than black rectangle. I understand however that it may be problem that user does not notice that the cut was applied. If we find 2-98% as bad as bad min/max we should invent something better, not just switch from one bad solution to another one. We can discuss for example the range (2-98%,1-99%,0.1-99.9%...) and different situations (data types, number of bands). Can you describe some examples where 2-98% is a problem (data type, number of bands, map content, features/phenomena represented by those 2+2%,...) so that we can think about it better? Radim > All the best. > - -- > Paolo Cavallini - www.faunalia.eu > QGIS & PostGIS courses: http://www.faunalia.eu/training.html > -BEGIN PGP SIGNATURE- > Version: GnuPG v1.4.15 (GNU/Linux) > Comment: Using GnuPG with Icedove - http://www.enigmail.net/ > > iEYEARECAAYFAlKqy3gACgkQ/NedwLUzIr6zmACfXL/qTHouaIHDyTxZP6CPMxVS > B10An2qhIuojWaPsCXnJmhb7stUbvQ6c > =I+Xp > -END PGP SIGNATURE- > ___ > Qgis-developer mailing list > Qgis-developer@lists.osgeo.org > http://lists.osgeo.org/mailman/listinfo/qgis-developer ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [Qgis-developer] min/max for rasters
>> Hi all. >> The default for raster rendering is to cut values at 2-98%. This is >> inappropriate in many cases, and confusing for users. Is that OK if I >> change the default to min/max? I also think that min/max would be more appropriate as default ina fresh installation. ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [Qgis-developer] QGIS Multi-threaded Rendering
Hi Martin, I guess I am also running into the PostgreSQL issue. In general my PostgreSQL based projects with many layers freeze after a very short time, while the SpatiaLite based projects work very nicely. Too many PostgreSQL connections? Can we limit the PostgreSQL connections or can you better re-use existing connections? Looks very promising - looking forward to testing my PostgreSQL based projects once this issue is fixed. Andreas Am 12.12.2013 16:46, schrieb Denis Rouzaud: > Hi Martin, > > So nice to see this close to master! > > Just tried with a big project. > > At launch, I have the database login showing up (since there is too many > connections apparently): > > And it doesn't stop showing up while in the project > > Will try with a smaller one ;) > > > Cheers, > Denis > ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [Qgis-developer] min/max for rasters
Hi On Fri, Dec 13, 2013 at 10:55 AM, Paolo Cavallini wrote: > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA1 > > Hi all. > The default for raster rendering is to cut values at 2-98%. This is > inappropriate in many cases, and confusing for users. Is that OK if I > change the default to min/max? > On your own system of in the source tree? I would prefer to keep the 2-98% in the source tree myself. Regards Tim > All the best. > - -- > Paolo Cavallini - www.faunalia.eu > QGIS & PostGIS courses: http://www.faunalia.eu/training.html > -BEGIN PGP SIGNATURE- > Version: GnuPG v1.4.15 (GNU/Linux) > Comment: Using GnuPG with Icedove - http://www.enigmail.net/ > > iEYEARECAAYFAlKqy3gACgkQ/NedwLUzIr6zmACfXL/qTHouaIHDyTxZP6CPMxVS > B10An2qhIuojWaPsCXnJmhb7stUbvQ6c > =I+Xp > -END PGP SIGNATURE- > ___ > Qgis-developer mailing list > Qgis-developer@lists.osgeo.org > http://lists.osgeo.org/mailman/listinfo/qgis-developer > -- Tim Sutton - QGIS Project Steering Committee Member == Please do not email me off-list with technical support questions. Using the lists will gain more exposure for your issues and the knowledge surrounding your issue will be shared with all. Irc: timlinux on #qgis at freenode.net == ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [Qgis-developer] QT 5.2 is out
On Fri, Dec 13, 2013 at 11:14 AM, Matthias Kuhn wrote: > I think Marco already has access to the GPS on his Android port. I > don't know how he did it, but I remember, that he has been showing it. Probabily he is using the native android functions. ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [Qgis-developer] QGIS Multi-threaded Rendering
Oh! One other thing that would be nice to keep in mind is the future support for mutliple canvases in QGIS. Regards Tim -- Tim Sutton - QGIS Project Steering Committee Member == Please do not email me off-list with technical support questions. Using the lists will gain more exposure for your issues and the knowledge surrounding your issue will be shared with all. Irc: timlinux on #qgis at freenode.net == ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [Qgis-developer] QGIS Multi-threaded Rendering
Hey On Thu, Dec 12, 2013 at 5:50 PM, Martin Dobias wrote: > Hi Tim! > > > > That all sounds absolutely brilliant! Thanks for such a nice clear > > description of how it all fits together. I know you are only considering > > layer-by-layer rendering but does your design accommodate further future > > optimisations easily? I'm thinking of things like: > > > > * predictive / off screen rendering of 3x3 canvas dimensions after the > > initial render so that any pan is near instantaneous (and would trigger a > > new off-screen render) > > I have had this idea in my mind while working on the project. In > theory map canvas can spawn several renderer jobs (instead of just > one) and let them render just one tile of the map. Some special > handling of the labeling would be necessary if we wanted labels that > are allowed to cross the border of tiles. > > Yeah - Larry and I had chatted before about using the AOI extents as a fake line feature with a 'may not be covered' rule in labelling - same for map viewframe when printing. I don't know if that is still on his agenda but it would be very nice to have. > > * on zoom, resample first then over render the resampled image (like open > > layers and other web toolkits do so you see a resampled version of the > old > > render which gets overpainted as the new render comes in) > > Actually that's already there :-) > When you change the extent, the old map will be still present, just > scaled. It will be there until the first update of the preview (now > the default is set to 250ms). So, first quarter of the second you will > be looking at the older resampled map (of course if your new map is > rendered under 250ms, it will be shown as soon as it is done). > > Cool. And if you have the 3x3 AOI buffer you could resample the whole thing when zooming out and the user would have something quite pleasant to look at while waiting for a redraw. > > * symbol layer render in threads - I believe even single layer draws can > > benefit greatly from the render-then-composite approach you are taking - > > rendering each feature into a buffer when symbol layers are enabled (and > > then compositing the buffers after rendering them) means that no feature > > should need to be retrieved more than once when rendering. > > This could be quite interesting thing to do. I guess the vector layer > renderer class could actually spawn further rendering tasks for each > symbol level after initial load of features and their division into > groups based on their symbol. > > > * render caching of symbol layers (so that if only one layer gets changed > > not all others need to be re-rendered) > > So you mean render caching not only the whole map layers, but also > levels within layers? I am not sure how big the benefit of introducing > that would be - if the added code complexity wouldn't outweigh the > benefit of optimizing such case (i.e. how much of user's total time in > QGIS does he/she spend waiting for update after a change to a symbol > layer?) > > Yeah this admittedly the weakest of the ideas I listed :-) > > * progressive rendering (e.g. rendering in a generalised way ala A > Huarte's > > patches and then update the display and the start a second pass render > with > > full detail) - that way you get a very fast first render but if you stick > > around at that AOI the render quality improves as the second pass happens > > I haven't really thought about that so far. We would probably need to > employ some guessing to know when does it make sense to actually do > the progressive rendering and when to render just once... > > This 'progressive' thing reminds me of another thing worth trying: by > default we could try to cache all reasonably sized vector datasets > upon their load into memory. Then we would render the layers without > actually querying the backend. When the rendering is done, we could > then optionally run the query on backend to check whether our cache > needs updating. > Yeah that would be awesome. > > Also, it may be interesting to build some temporary overviews for > raster layers (if not present) and do the progressive rendering there > - first a low quality render from the overview, then a real rendered > image. > > > Yeah that also sounds good! Thanks again for making all this threading stuff happen! Regards Tim > > I know there are possible issues with memory consumption with some of the > > above ideas, and they are definitely not on your current roadmap, but it > > would be good to at least play a little mental soccer with the above > ideas > > and see if the architecture you have devised can accommodate such further > > optimisations cleanly in the future. > > Sure, it is always good to think ahead about possible improvements! > > Cheers > Martin > -- Tim Sutton - QGIS Project Steering Committee Member == Please do not email me off-list with technical support questions. Using the lists will gain mo
Re: [Qgis-developer] QGIS Multi-threaded Rendering
On Fri, Dec 13, 2013 at 2:49 PM, Marco Hugentobler wrote: > Hi Martin > > Wow, nice work! > After first testing, it works very nice. I hope you can merge the branch > quite soon. What is currently missing in order to make the merge? Hi Marco as mentioned in my earlier mail, it's mainly that not all providers have been updated yet. Then, I haven't checked how the server behaves and the point displacement renderer does not work right now. There are few other unresolved things like geometry cache for editable layer, joined layers, SLD with pixel units, split extent in renderer. Martin ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [Qgis-developer] QT 5.2 is out
On Fre 13 Dez 2013 11:10:11 CET, G. Allegri wrote: > > It lacks some important, tipycal mobile features, like > positioning/satellite information, bluetooth, etc., which are planned > for 5.3.0. > I think Marco already has access to the GPS on his Android port. I don't know how he did it, but I remember, that he has been showing it. ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [Qgis-developer] QT 5.2 is out
> > > Qt is the project that now includes Android support - with open source > license as usual. > > Qt Mobile is a (new) different product, it comes with commercial > license, and brings you additional tools + support + cloud service. > You do not need Qt Mobile for QGIS on Android. > Thanks Martin for claryfing it. I've given a look to the port status and it seems quite good. It lacks some important, tipycal mobile features, like positioning/satellite information, bluetooth, etc., which are planned for 5.3.0. giovanni > > Martin > -- Giovanni Allegri http://about.me/giovanniallegri blog: http://blog.spaziogis.it GEO+ geomatica in Italia http://bit.ly/GEOplus ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [Qgis-developer] fTools: stratified random selection broken?
> Hi all. > Apparently the tool is borken on 2.0.1: anyone confirms? yes http://hub.qgis.org/issues/8864 ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [Qgis-developer] QT 5.2 is out
On Fri, Dec 13, 2013 at 4:13 PM, Denis Rouzaud wrote: >> >> Yeah, it would be great to see QGIS ported to Qt5. My hope is that >> QGIS 3.0 will use Qt5 and Python3. >> >> From what I have read in Qt blogs, Qt 5.2 should really make >> development for Android easier than ever before. > > Is there any guidance regarding this? > I mean, on one hand, I think users (plugins dev mainly) might be upset if > this change is happening so close to 2.0 release and on the other hand we > don't want to wait ten years for this to happen, especially if this brings > QGIS more easily on mobile device. > If QGIS 3 is brought as the "mobile" release, it might sweeten the pill. We should be able to get our codebase into a state where it will compile with both Qt4 and Qt5. The transition should be relatively painless (mostly fixing the build and plugin/provider system). Even after that, all the 2.x releases would still be based on Qt4 and Python2 - no change whatsoever, so no reason for making plugin developers angry. Just people building their custom projects based on QGIS would eventually be able to use the new features from Qt5. Martin ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [Qgis-developer] QT 5.2 is out
Hi, On 13. 12. 13 09:57, Martin Dobias wrote: Hi Luca On Fri, Dec 13, 2013 at 2:13 PM, Luca Manganelli wrote: Hi, as some of you have noticed, QT 5.2 is out. From release notes [1]: "I am proud to say that Qt 5.2 fully brings Qt into the mobile space as a true player in the app development market supporting Android, iOS, BlackBerry, Sailfish/Jolla and Ubuntu Mobile." Yeah, it would be great to see QGIS ported to Qt5. My hope is that QGIS 3.0 will use Qt5 and Python3. From what I have read in Qt blogs, Qt 5.2 should really make development for Android easier than ever before. Is there any guidance regarding this? I mean, on one hand, I think users (plugins dev mainly) might be upset if this change is happening so close to 2.0 release and on the other hand we don't want to wait ten years for this to happen, especially if this brings QGIS more easily on mobile device. If QGIS 3 is brought as the "mobile" release, it might sweeten the pill. Now it could be easier to port QGis to these platforms. I remember that QT, from 5.0, uses OpenGL backend and it could render maps faster than before. Not to my knowledge. OpenGL backend is used for QML-based user interfaces (Qt Quick 2). But we are not using that for map rendering. Switching to OpenGL for map rendering would be a big project. Regards Martin ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [Qgis-developer] QT 5.2 is out
On Fri, Dec 13, 2013 at 3:50 PM, G. Allegri wrote: > Hi Luca, AFAIK Qt Mobile is available only under proprietary and commercial > license. Am I wrong? Qt is the project that now includes Android support - with open source license as usual. Qt Mobile is a (new) different product, it comes with commercial license, and brings you additional tools + support + cloud service. You do not need Qt Mobile for QGIS on Android. Martin ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [Qgis-developer] QT 5.2 is out
Hi Luca On Fri, Dec 13, 2013 at 2:13 PM, Luca Manganelli wrote: > Hi, > > as some of you have noticed, QT 5.2 is out. From release notes [1]: > > "I am proud to say that Qt 5.2 fully brings Qt into the mobile space as a > true player in the app development market supporting Android, iOS, > BlackBerry, Sailfish/Jolla and Ubuntu Mobile." Yeah, it would be great to see QGIS ported to Qt5. My hope is that QGIS 3.0 will use Qt5 and Python3. >From what I have read in Qt blogs, Qt 5.2 should really make development for Android easier than ever before. > Now it could be easier to port QGis to these platforms. I remember that QT, > from 5.0, uses OpenGL backend and it could render maps faster than before. Not to my knowledge. OpenGL backend is used for QML-based user interfaces (Qt Quick 2). But we are not using that for map rendering. Switching to OpenGL for map rendering would be a big project. Regards Martin ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [Qgis-developer] QGIS Multi-threaded Rendering
> I believe GeoServer (well, GeoWebCache) uses "metatiling" for that purpose within its WMTS/TMS. My understanding is that rather than rendering a single 256*256 pixel tile like it was asked to, it renders a grid of 4*4 (adjustable; but that's the default) of those tiles (so 1024*1024 pixels) and then clips that to get the requested tile. The result is that labels look quite good crossing tile borders. Maybe something similar could work for QGIS. Metatiling is a trick emplyed by almost every caching/tiling system. It's on the client side. While buffering is made by the server. Using both of them give the best results, but they're not alternatives. giovanni > Jonathan > > > On 12 December 2013 16:23, Bernhard Ströbl wrote: >> >> Hi Martin, >> >> Am 12.12.2013 16:50, schrieb Martin Dobias: >> >>> Hi Tim! >>> >>> That all sounds absolutely brilliant! Thanks for such a nice clear description of how it all fits together. I know you are only considering layer-by-layer rendering but does your design accommodate further future optimisations easily? I'm thinking of things like: * predictive / off screen rendering of 3x3 canvas dimensions after the initial render so that any pan is near instantaneous (and would trigger a new off-screen render) >>> >>> >>> I have had this idea in my mind while working on the project. In >>> theory map canvas can spawn several renderer jobs (instead of just >>> one) and let them render just one tile of the map. Some special >>> handling of the labeling would be necessary if we wanted labels that >>> are allowed to cross the border of tiles. >>> >> >> that reminds me that I needed some way to keep a stripe around the edge free of labels in QGIS server for tiling purposes (labels were cut) >> if you address this issue it would be nice to have this somehow customizable for QGIS server. >> >> Bernhard >> >> >> __ Information from ESET Mail Security, version of virus signature database 9165 (20131212) __ >> >> The message was checked by ESET Mail Security. >> http://www.eset.com >> >> >> >> ___ >> Qgis-developer mailing list >> Qgis-developer@lists.osgeo.org >> http://lists.osgeo.org/mailman/listinfo/qgis-developer > > > > This transmission is intended for the named addressee(s) only and may contain sensitive or protectively marked material up to RESTRICTED and should be handled accordingly. Unless you are the named addressee (or authorised to receive it for the addressee) you may not copy or use it, or disclose it to anyone else. If you have received this transmission in error please notify the sender immediately. All email traffic sent to or from us, including without limitation all GCSX traffic, may be subject to recording and/or monitoring in accordance with relevant legislation. > > ___ > Qgis-developer mailing list > Qgis-developer@lists.osgeo.org > http://lists.osgeo.org/mailman/listinfo/qgis-developer ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer
[Qgis-developer] min/max for rasters
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hi all. The default for raster rendering is to cut values at 2-98%. This is inappropriate in many cases, and confusing for users. Is that OK if I change the default to min/max? All the best. - -- Paolo Cavallini - www.faunalia.eu QGIS & PostGIS courses: http://www.faunalia.eu/training.html -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.15 (GNU/Linux) Comment: Using GnuPG with Icedove - http://www.enigmail.net/ iEYEARECAAYFAlKqy3gACgkQ/NedwLUzIr6zmACfXL/qTHouaIHDyTxZP6CPMxVS B10An2qhIuojWaPsCXnJmhb7stUbvQ6c =I+Xp -END PGP SIGNATURE- ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [Qgis-developer] QT 5.2 is out
Hi Luca, AFAIK Qt Mobile is available only under proprietary and commercial license. Am I wrong? giovanni Il 13/dic/2013 08:13 "Luca Manganelli" ha scritto: > Hi, > > as some of you have noticed, QT 5.2 is out. From release notes [1]: > > "I am proud to say that Qt 5.2 fully brings Qt into the mobile space as a > true player in the app development market supporting Android, iOS, > BlackBerry, Sailfish/Jolla and Ubuntu Mobile." > > Now it could be easier to port QGis to these platforms. I remember that > QT, from 5.0, uses OpenGL backend and it could render maps faster than > before. > > > [1] > http://blog.qt.digia.com/blog/2013/12/12/qt-5-2-released-the-best-qt-yet/ > > > > ___ > Qgis-developer mailing list > Qgis-developer@lists.osgeo.org > http://lists.osgeo.org/mailman/listinfo/qgis-developer > ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer
