Re: [Qgis-developer] min/max for rasters
On Mon, Dec 16, 2013 at 8:41 AM, Vincent Schut sc...@sarvision.nl wrote: Problems of course arise when people/programs do not correctly set nodata values, etc. (e.g. in the case of rasterizing sparse vectors, all other pixels should be nodata). I'd say, do not build in too much intelligence to correct for things like that, in the end it only makes things more complicated. +1 I feel very much like Vincent. Maybe there is potential for improvements in NULL value detection? Best wishes, Anita ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [Qgis-developer] min/max for rasters
If there is some decision taken by the software, should it be communicated to the user? I am thinking of a message bar popping up Hey, I think that a 2-98% representation suits your new layer best and I've applied that for you. But if you want to change it: here is a combobox and a link to the layer properties. Matthias On Mon 16 Dez 2013 09:22:16 CET, Anita Graser wrote: On Mon, Dec 16, 2013 at 8:41 AM, Vincent Schut sc...@sarvision.nl wrote: Problems of course arise when people/programs do not correctly set nodata values, etc. (e.g. in the case of rasterizing sparse vectors, all other pixels should be nodata). I'd say, do not build in too much intelligence to correct for things like that, in the end it only makes things more complicated. +1 I feel very much like Vincent. Maybe there is potential for improvements in NULL value detection? Best wishes, Anita ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [Qgis-developer] min/max for rasters
Il 16/12/2013 11:29, Matthias Kuhn ha scritto: If there is some decision taken by the software, should it be communicated to the user? I am thinking of a message bar popping up Hey, I think that a 2-98% representation suits your new layer best and I've applied that for you. But if you want to change it: here is a combobox and a link to the layer properties. this would be annoying, but I think we should not drop (even if virtually) values in any case without letting the user know. does anybody confirm the general option is not working? furthermore, I think some cleaning of the raster layer properties dialog would also help users. thanks. -- Paolo Cavallini - www.faunalia.eu QGIS PostGIS courses: http://www.faunalia.eu/training.html ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [Qgis-developer] min/max for rasters
Il 16/12/2013 11:36, Paolo Cavallini ha scritto: this would be annoying, but I think we should not drop (even if BTW, it would be useful to review how other GIS/RS sw deals with this problem. -- Paolo Cavallini - www.faunalia.eu QGIS PostGIS courses: http://www.faunalia.eu/training.html ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [Qgis-developer] min/max for rasters
On Sat, Dec 14, 2013 at 7:17 AM, Paolo Cavallini cavall...@faunalia.it wrote: Il 13/12/2013 20:18, Radim Blazek ha scritto: Can you describe some examples where 2-98% is a problem (data type, number of bands, map content, features/phenomena represented by those 2+2%,...) so that we can think about it better? Example #1 (less problematic): dtm and their legend are always shown wrong; newbies do not understand why Example #2 (more serious): rasterizing sparse vectors (e.g. rivers) results in a black rectangle, as the number of pixels with valid data is 2%. We can try to distinguish discrete from continuous data by number of values, but what will be the threshold? 2, 10, 50, 100...? In fact, I think we should help users more, e.g. by applying non linear colour scaling (log, exp) in case of very skewed raster values distribution: if data are more or less normally distributed, no cut is applied, and linear scaling is used; if they are badly skewdw or with outliers, apply a non linear colour scaling. With some thinking, this should solve most if not all user cases, without asking a normal user to understand much about raster stats. Can you define precisely more or less normally distributed? In general, applying more sophisticated/sensitive decision can catch correctly more cases but it will become more difficult to be communicated to a user. Until we hide it under single intelligent style option. However, in my case the general setting use min/max does not seem to be working. Contrast enhancement Stretch to min/max is not applied to new raster layer? For me it works. Radim Thanks for your thoughts. -- Paolo Cavallini - www.faunalia.eu QGIS PostGIS courses: http://www.faunalia.eu/training.html ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [Qgis-developer] min/max for rasters
On Mon, Dec 16, 2013 at 8:41 AM, Vincent Schut sc...@sarvision.nl wrote: On 12/14/13 07:17, Paolo Cavallini wrote: Il 13/12/2013 20:18, Radim Blazek ha scritto: Can you describe some examples where 2-98% is a problem (data type, number of bands, map content, features/phenomena represented by those 2+2%,...) so that we can think about it better? Example #1 (less problematic): dtm and their legend are always shown wrong; newbies do not understand why Example #2 (more serious): rasterizing sparse vectors (e.g. rivers) results in a black rectangle, as the number of pixels with valid data is 2%. In fact, I think we should help users more, e.g. by applying non linear colour scaling (log, exp) in case of very skewed raster values distribution: if data are more or less normally distributed, no cut is applied, and linear scaling is used; if they are badly skewdw or with outliers, apply a non linear colour scaling. With some thinking, this should solve most if not all user cases, without asking a normal user to understand much about raster stats. However, in my case the general setting use min/max does not seem to be working. Thanks for your thoughts. Imho, a lot could be derived from the image metadata: datatype, number of bands, 'photographic interpretation', etc. - for 3 bands, 8 bit, assume rgb and do not stretch - for 4 bands, 8 bit, assume rgba and do not stretch - more bands and/or datatypes 8bit or float: usually means satellite imagery. I'd say, start with a stretch of 2-98%. Users of satellite imagery mostly know what to do when their image then still appears black, I think? - 1 band data, 8bit: probably conitnuous data, possibly a dtm/dem, apply a stretch, maybe also 2-98 %? Of course all nodata pixels should be excluded from the 2-98% calculations. - 1 band data, 8bit: probably sparse or 'class' data: discrete, might have lots of actual nodata values. A good default strategy might be to apply a random or default colormap. Have you tried to run QGIS 2.0? I think that most of what you are suggesting is default and you can change the defaults if you want in: Settings Options Rendering Rasters Missing is identification of discreet data. Personally, I would refrain from non-linear stretch types as default. When I open a raster, I do not want too much 'intelligence' to happen. Also, I usually find the x-y% type of stretching more useful than the -x +y stdev type of streching, but that might have to do with the datasets I usually work with. Problems of course arise when people/programs do not correctly set nodata values, etc. (e.g. in the case of rasterizing sparse vectors, all other pixels should be nodata). I'd say, do not build in too much intelligence to correct for things like that, in the end it only makes things more complicated. I also hate programs which intelligently do something what I don't want. OTOH I also hate application which are not able to do some very simple task like render a raster so that I can see more than black rectangle. Unfortunately such simple task appears to be quite complicated. In this situation I would prefer first to try to improve style decision instead of resign completely. Radim My 2 cents. Best, Vincent Schut. ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [Qgis-developer] min/max for rasters
On Mon, Dec 16, 2013 at 9:29 AM, Matthias Kuhn matthias.k...@gmx.ch wrote: If there is some decision taken by the software, Decisions are always taken in case of rasters. We are only talking about particular detail of that decision. I.e. we have to choose renderer, decide if stretch should be applied or not and stretching mode is the last. What has to be communicated and what not? should it be communicated to the user? I am thinking of a message bar popping up Hey, I think that a 2-98% representation suits your new layer best and I've applied that for you. But if you want to change it: here is a combobox and a link to the layer properties. Communicated yes, but how? Dialog is disturbing, soft message are not being read. Radim Matthias On Mon 16 Dez 2013 09:22:16 CET, Anita Graser wrote: On Mon, Dec 16, 2013 at 8:41 AM, Vincent Schut sc...@sarvision.nl wrote: Problems of course arise when people/programs do not correctly set nodata values, etc. (e.g. in the case of rasterizing sparse vectors, all other pixels should be nodata). I'd say, do not build in too much intelligence to correct for things like that, in the end it only makes things more complicated. +1 I feel very much like Vincent. Maybe there is potential for improvements in NULL value detection? Best wishes, Anita ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [Qgis-developer] min/max for rasters
On Sat, Dec 14, 2013 at 7:57 AM, aperi2007 aperi2...@gmail.com wrote: AFAIK The strategy of 2-98 is usually useful when there a noised image, because we assume that the noise is a white noise and it is randomized an isolated spikes. THis is absolutely a right theory and really useful, ma what kind of imagge are usually used in a gis system. If we think at the ortophoto image the noise could be really happened because they came from a photo-sensor. But is we think to a artificial image, like the 2-colors balck-white images named carta tecnica thata are trasposition of vectorial data. Them are no noised images and has a really thin lines. Also the artifical thematic chars with colors and point and symbols and lines (outline and so on) are noise-less images. Don't forget to think also to geological charts. Are all noise-less images. So what kind of image are more used in a GIS system ? This is not simple question. The response is , it is dependent by the kind of work you should do.. Yes, so we can either ask user when QGIS is run the first time to set some global defaults (nobody will do (correctly)) or we can try to guess the type of data which will never be 100% but it can be better than it is now. But also another question is: Usually the ortophoto are not simple to have . They are produced and have a license. The thematic images are more easy to produce and are often without a license or has a free license. More often the ortophoto images are available from a WMS system, and this is a solution that deny the use of the 2-98 strategy. BTW, we can reconsider to decompose WMS images to RGB bands. Radim Andrea. On 14/12/2013 07:17, Paolo Cavallini wrote: Il 13/12/2013 20:18, Radim Blazek ha scritto: Can you describe some examples where 2-98% is a problem (data type, number of bands, map content, features/phenomena represented by those 2+2%,...) so that we can think about it better? Example #1 (less problematic): dtm and their legend are always shown wrong; newbies do not understand why Example #2 (more serious): rasterizing sparse vectors (e.g. rivers) results in a black rectangle, as the number of pixels with valid data is 2%. In fact, I think we should help users more, e.g. by applying non linear colour scaling (log, exp) in case of very skewed raster values distribution: if data are more or less normally distributed, no cut is applied, and linear scaling is used; if they are badly skewdw or with outliers, apply a non linear colour scaling. With some thinking, this should solve most if not all user cases, without asking a normal user to understand much about raster stats. However, in my case the general setting use min/max does not seem to be working. Thanks for your thoughts. ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [Qgis-developer] min/max for rasters
What would be nice is if QGIS has a trouble shooting option under the help menu. Through a series of questions, the trouble shooting system can give the user suggestions on solving their problem. -Bob On Dec 16, 2013 2:46 AM, Paolo Cavallini cavall...@faunalia.it wrote: Il 16/12/2013 12:30, Radim Blazek ha scritto: Can you define precisely more or less normally distributed? I have to study a bit to be more accurate. Perhaps we can use some outlier detection algs (I remember in R they have implemented a few). In general, applying more sophisticated/sensitive decision can catch correctly more cases but it will become more difficult to be communicated to a user. Until we hide it under single intelligent style option. IMHO normal users should ideally have something reasonable displayed out of the box, at least in most cases. Power users can tweak things as they want, I do not think they should be our first target. However, in my case the general setting use min/max does not seem to be working. Contrast enhancement Stretch to min/max is not applied to new raster layer? For me it works. Not, is the Limits - Minimum/Maximum that has a behavious I did not understand at first (I assumed it used 100% by default). Sorry about the noise. All the best. -- Paolo Cavallini - www.faunalia.eu QGIS PostGIS courses: http://www.faunalia.eu/training.html ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [Qgis-developer] min/max for rasters
On 12/14/13 07:17, Paolo Cavallini wrote: Il 13/12/2013 20:18, Radim Blazek ha scritto: Can you describe some examples where 2-98% is a problem (data type, number of bands, map content, features/phenomena represented by those 2+2%,...) so that we can think about it better? Example #1 (less problematic): dtm and their legend are always shown wrong; newbies do not understand why Example #2 (more serious): rasterizing sparse vectors (e.g. rivers) results in a black rectangle, as the number of pixels with valid data is 2%. In fact, I think we should help users more, e.g. by applying non linear colour scaling (log, exp) in case of very skewed raster values distribution: if data are more or less normally distributed, no cut is applied, and linear scaling is used; if they are badly skewdw or with outliers, apply a non linear colour scaling. With some thinking, this should solve most if not all user cases, without asking a normal user to understand much about raster stats. However, in my case the general setting use min/max does not seem to be working. Thanks for your thoughts. Imho, a lot could be derived from the image metadata: datatype, number of bands, 'photographic interpretation', etc. - for 3 bands, 8 bit, assume rgb and do not stretch - for 4 bands, 8 bit, assume rgba and do not stretch - more bands and/or datatypes 8bit or float: usually means satellite imagery. I'd say, start with a stretch of 2-98%. Users of satellite imagery mostly know what to do when their image then still appears black, I think? - 1 band data, 8bit: probably conitnuous data, possibly a dtm/dem, apply a stretch, maybe also 2-98 %? Of course all nodata pixels should be excluded from the 2-98% calculations. - 1 band data, 8bit: probably sparse or 'class' data: discrete, might have lots of actual nodata values. A good default strategy might be to apply a random or default colormap. Personally, I would refrain from non-linear stretch types as default. When I open a raster, I do not want too much 'intelligence' to happen. Also, I usually find the x-y% type of stretching more useful than the -x +y stdev type of streching, but that might have to do with the datasets I usually work with. Problems of course arise when people/programs do not correctly set nodata values, etc. (e.g. in the case of rasterizing sparse vectors, all other pixels should be nodata). I'd say, do not build in too much intelligence to correct for things like that, in the end it only makes things more complicated. My 2 cents. Best, Vincent Schut. ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer
[Qgis-developer] min/max for rasters
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hi all. The default for raster rendering is to cut values at 2-98%. This is inappropriate in many cases, and confusing for users. Is that OK if I change the default to min/max? All the best. - -- Paolo Cavallini - www.faunalia.eu QGIS PostGIS courses: http://www.faunalia.eu/training.html -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.15 (GNU/Linux) Comment: Using GnuPG with Icedove - http://www.enigmail.net/ iEYEARECAAYFAlKqy3gACgkQ/NedwLUzIr6zmACfXL/qTHouaIHDyTxZP6CPMxVS B10An2qhIuojWaPsCXnJmhb7stUbvQ6c =I+Xp -END PGP SIGNATURE- ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [Qgis-developer] min/max for rasters
Hi On Fri, Dec 13, 2013 at 10:55 AM, Paolo Cavallini cavall...@faunalia.itwrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hi all. The default for raster rendering is to cut values at 2-98%. This is inappropriate in many cases, and confusing for users. Is that OK if I change the default to min/max? On your own system of in the source tree? I would prefer to keep the 2-98% in the source tree myself. Regards Tim All the best. - -- Paolo Cavallini - www.faunalia.eu QGIS PostGIS courses: http://www.faunalia.eu/training.html -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.15 (GNU/Linux) Comment: Using GnuPG with Icedove - http://www.enigmail.net/ iEYEARECAAYFAlKqy3gACgkQ/NedwLUzIr6zmACfXL/qTHouaIHDyTxZP6CPMxVS B10An2qhIuojWaPsCXnJmhb7stUbvQ6c =I+Xp -END PGP SIGNATURE- ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer -- Tim Sutton - QGIS Project Steering Committee Member == Please do not email me off-list with technical support questions. Using the lists will gain more exposure for your issues and the knowledge surrounding your issue will be shared with all. Irc: timlinux on #qgis at freenode.net == ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [Qgis-developer] min/max for rasters
Hi all. The default for raster rendering is to cut values at 2-98%. This is inappropriate in many cases, and confusing for users. Is that OK if I change the default to min/max? I also think that min/max would be more appropriate as default ina fresh installation. ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [Qgis-developer] min/max for rasters
On Fri, Dec 13, 2013 at 9:55 AM, Paolo Cavallini cavall...@faunalia.it wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hi all. The default for raster rendering is to cut values at 2-98%. This is inappropriate in many cases, and confusing for users. Is that OK if I change the default to min/max? Min/max is inappropriate in many other cases. It is quite common to get data with few outliers. The result in such cases with min/max is black rectangle which is very frustrating and that was the case with 1.8. I believe that it is better to get inappropriately rendered picture than black rectangle. I understand however that it may be problem that user does not notice that the cut was applied. If we find 2-98% as bad as bad min/max we should invent something better, not just switch from one bad solution to another one. We can discuss for example the range (2-98%,1-99%,0.1-99.9%...) and different situations (data types, number of bands). Can you describe some examples where 2-98% is a problem (data type, number of bands, map content, features/phenomena represented by those 2+2%,...) so that we can think about it better? Radim All the best. - -- Paolo Cavallini - www.faunalia.eu QGIS PostGIS courses: http://www.faunalia.eu/training.html -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.15 (GNU/Linux) Comment: Using GnuPG with Icedove - http://www.enigmail.net/ iEYEARECAAYFAlKqy3gACgkQ/NedwLUzIr6zmACfXL/qTHouaIHDyTxZP6CPMxVS B10An2qhIuojWaPsCXnJmhb7stUbvQ6c =I+Xp -END PGP SIGNATURE- ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [Qgis-developer] min/max for rasters
Il 13/12/2013 20:18, Radim Blazek ha scritto: Can you describe some examples where 2-98% is a problem (data type, number of bands, map content, features/phenomena represented by those 2+2%,...) so that we can think about it better? Example #1 (less problematic): dtm and their legend are always shown wrong; newbies do not understand why Example #2 (more serious): rasterizing sparse vectors (e.g. rivers) results in a black rectangle, as the number of pixels with valid data is 2%. In fact, I think we should help users more, e.g. by applying non linear colour scaling (log, exp) in case of very skewed raster values distribution: if data are more or less normally distributed, no cut is applied, and linear scaling is used; if they are badly skewdw or with outliers, apply a non linear colour scaling. With some thinking, this should solve most if not all user cases, without asking a normal user to understand much about raster stats. However, in my case the general setting use min/max does not seem to be working. Thanks for your thoughts. -- Paolo Cavallini - www.faunalia.eu QGIS PostGIS courses: http://www.faunalia.eu/training.html ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [Qgis-developer] min/max for rasters
AFAIK The strategy of 2-98 is usually useful when there a noised image, because we assume that the noise is a white noise and it is randomized an isolated spikes. THis is absolutely a right theory and really useful, ma what kind of imagge are usually used in a gis system. If we think at the ortophoto image the noise could be really happened because they came from a photo-sensor. But is we think to a artificial image, like the 2-colors balck-white images named carta tecnica thata are trasposition of vectorial data. Them are no noised images and has a really thin lines. Also the artifical thematic chars with colors and point and symbols and lines (outline and so on) are noise-less images. Don't forget to think also to geological charts. Are all noise-less images. So what kind of image are more used in a GIS system ? This is not simple question. The response is , it is dependent by the kind of work you should do.. But also another question is: Usually the ortophoto are not simple to have . They are produced and have a license. The thematic images are more easy to produce and are often without a license or has a free license. More often the ortophoto images are available from a WMS system, and this is a solution that deny the use of the 2-98 strategy. Andrea. On 14/12/2013 07:17, Paolo Cavallini wrote: Il 13/12/2013 20:18, Radim Blazek ha scritto: Can you describe some examples where 2-98% is a problem (data type, number of bands, map content, features/phenomena represented by those 2+2%,...) so that we can think about it better? Example #1 (less problematic): dtm and their legend are always shown wrong; newbies do not understand why Example #2 (more serious): rasterizing sparse vectors (e.g. rivers) results in a black rectangle, as the number of pixels with valid data is 2%. In fact, I think we should help users more, e.g. by applying non linear colour scaling (log, exp) in case of very skewed raster values distribution: if data are more or less normally distributed, no cut is applied, and linear scaling is used; if they are badly skewdw or with outliers, apply a non linear colour scaling. With some thinking, this should solve most if not all user cases, without asking a normal user to understand much about raster stats. However, in my case the general setting use min/max does not seem to be working. Thanks for your thoughts. ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer
