Re: [GRASS-user] GRASS earth curvature correction (viewsheds, LOS)
Hamish wrote: [...] the ideal solution is to have both curvature and refection corrections implemented as flags in the new improved r.viewshed. [...] Benjamin wrote: That should be really easy to do. All that's needed is to amend the existing correction but taking away 1/7 to account for the adverse effect of refraction. ok, done for r.viewshed in r46423. Number of visible cells reduces slightly when the curvature flag is used, and rebounds ever so slightly when the refraction flag is used. Please test. Well, that refraction correction is really a rough simplification of reality. Essentially, it uses the same amount of correction as ArcGIS. There is some justification for this. You can find links to articles here: http://mapaspects.org/content/effects-curvature-earth-refraction-light-air-and-fuzzy-viewsheds-arcgis-92 I could not get at the Yoeli(1985) article as it's behind a paywall my univ does not subscribe to. Can anyone say what's in it? But in summary, accounting for realistic refraction conditions would be much more complex, as it would also have to take into plus different refraction at different elevations, etc. I don't mind that / it is not so different from the physics I do in my day job, and just using a fudge factor of +1/7th leaves me feeling like the job is poorly done. Passing the coeff off to the user without further guidance seems like a bit of a cop out. I suppose there is a gradient in the coeff as you move from the tropics to high latitudes, daily temperature, Linke factor, humidity, aerosols, etc ... ? But given that most DEMs have an inherent vertical error that is greater than the influence of these effects, can we quantify that? for example what's STRM 95% confidence accuracy? I am not sure it's worth spending too much time on (it might be for very long distance visibility -- I just don't know). it would be good for us to do a rough back of the envelope calc to justify that before fully forgetting about it. I guess for the worst case scenario we could try the views from Mt. Everest and/or Olympus Mons and see what difference it makes. thanks, Hamish ___ grass-user mailing list grass-user@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-user
Re: [GRASS-user] GRASS earth curvature correction (viewsheds, LOS)
On Thu, May 26, 2011 at 10:50 AM, Hamish hamis...@yahoo.com wrote: Hamish wrote: [...] the ideal solution is to have both curvature and refection corrections implemented as flags in the new improved r.viewshed. [...] Benjamin wrote: That should be really easy to do. All that's needed is to amend the existing correction but taking away 1/7 to account for the adverse effect of refraction. ok, done for r.viewshed in r46423. Number of visible cells reduces slightly when the curvature flag is used, and rebounds ever so slightly when the refraction flag is used. Please test. Well, that refraction correction is really a rough simplification of reality. Essentially, it uses the same amount of correction as ArcGIS. There is some justification for this. You can find links to articles here: http://mapaspects.org/content/effects-curvature-earth-refraction-light-air-and-fuzzy-viewsheds-arcgis-92 Hm-hm. Citing from the website: The problem is that the ratio of change due to air to curvature is not 1:7 (0.13), as the standard refraction coefficient suggests. It is 0.325. Last spring I put together an Excel sheet that computes this ratio. Having the adjustable details (altitude, air pressure, wavelength, etc.) did show me that the ratio never really changes (given earthly conditions). What it did show me was that the ratio was always 0.325. [...] But given that most DEMs have an inherent vertical error that is greater than the influence of these effects, can we quantify that? for example what's STRM 95% confidence accuracy? From Farr et al. 2007: Summary of SRTM performance. All quantities represent 90% errors in meters. Africa Australia Eurasia Islands N. America S. America Absolute Geolocation Error 11.97.2 8.8 9 12.69 Absolute Height Error 5.6 6 6.2 8 9 6.2 Relative Height Error 9.8 4.7 8.7 6.2 7 5.5 Long Wavelength Height Error3.1 6 2.6 3.7 4 4.9 [sorry for the ugly format, it's tab separated] I am not sure it's worth spending too much time on (it might be for very long distance visibility -- I just don't know). it would be good for us to do a rough back of the envelope calc to justify that before fully forgetting about it. I guess for the worst case scenario we could try the views from Mt. Everest and/or Olympus Mons and see what difference it makes. No need to go into mountains, just increase observer elevation offset, preferably in a moderately flat area to get really far views. Using correction for earth curvature only, max is a bit more than 100 km with 3km observation offset. 200km is impossible without leaving earth's atmosphere. Markus M ___ grass-user mailing list grass-user@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-user
Re: [GRASS-user] GRASS earth curvature correction (viewsheds, LOS)
Hm-hm. Citing from the website: The problem is that the ratio of change due to air to curvature is not 1:7 (0.13), as the standard refraction coefficient suggests. It is 0.325. As far as I can tell, this is a mis-understanding. The value 0.325 applies to radio waves. Visible light is very close to 1:7. You can read up on the arguments here: http://forums.esri.com/Thread.asp?c=3f=40t=161962#474778 There is also some more information in that forum thread regarding more realistic modelling of refraction under different atmospheric conditions. I realize the whole discourse is somewhat clouded. But I don't have access to most of the relevant literature for the time being, nor do I have the necessary scientific background -- so any fresh input will be much appreciated! Btw.: using r.ecurv.comp, one can freely specify the atmospheric correction factor. [...] But given that most DEMs have an inherent vertical error that is greater than the influence of these effects, can we quantify that? for example what's STRM 95% confidence accuracy? From Farr et al. 2007: Summary of SRTM performance. All quantities represent 90% errors in meters. Africa Australia Eurasia Islands N. America S. America Absolute Geolocation Error 11.9 7.2 8.8 9 12.6 9 Absolute Height Error 5.6 6 6.2 8 9 6.2 Relative Height Error 9.8 4.7 8.7 6.2 7 5.5 Long Wavelength Height Error 3.1 6 2.6 3.7 4 4.9 [sorry for the ugly format, it's tab separated] What I wold love to see is a method for probabilistic viewsheds, which adds random +/- offsets (within the vertical error range) to the elevation model cells, runs the viewshed computations several times, each time with new random offsets, then outputs the average visibility of each cell after n runs (not sure how best to determine n). That would be much more realistic than those over- confident 0/1 viewsheds. Such a method could even take into account roughly modelled blocks of vegetation or other obstacles for which height is hard to quantify precisely. -- shouldn't be too hard to implement in a little script. Ben I am not sure it's worth spending too much time on (it might be for very long distance visibility -- I just don't know). it would be good for us to do a rough back of the envelope calc to justify that before fully forgetting about it. I guess for the worst case scenario we could try the views from Mt. Everest and/or Olympus Mons and see what difference it makes. No need to go into mountains, just increase observer elevation offset, preferably in a moderately flat area to get really far views. Using correction for earth curvature only, max is a bit more than 100 km with 3km observation offset. 200km is impossible without leaving earth's atmosphere. Markus M -- Files attached to this email may be in ISO 26300 format (OASIS Open Document Format). If you have difficulty opening them, please visit http://iso26300.info for more information. ___ grass-user mailing list grass-user@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-user
Re: [GRASS-user] GRASS earth curvature correction (viewsheds, LOS)
- Original Message - Hamish wrote: [..] ok, done for r.viewshed in r46423. Number of visible cells reduces slightly when the curvature flag is used, and rebounds ever so slightly when the refraction flag is used. Please test. Cool, that's what one would expect. Reassuring. Well, that refraction correction is really a rough simplification of reality. Essentially, it uses the same amount of correction as ArcGIS. There is some justification for this. You can find links to articles here: http://mapaspects.org/content/effects-curvature-earth-refraction-light-air-and-fuzzy-viewsheds-arcgis-92 I could not get at the Yoeli(1985) article as it's behind a paywall my univ does not subscribe to. Can anyone say what's in it? But in summary, accounting for realistic refraction conditions would be much more complex, as it would also have to take into plus different refraction at different elevations, etc. I don't mind that / it is not so different from the physics I do in my day job, and just using a fudge factor of +1/7th leaves me feeling like the job is poorly done. Passing the coeff off to the user without further guidance seems like a bit of a cop out. I suppose there is a gradient in the coeff as you move from the tropics to high latitudes, daily temperature, Linke factor, humidity, aerosols, etc ... ? I am sure there is. But I lack the background to judge this correctly. But given that most DEMs have an inherent vertical error that is greater than the influence of these effects, can we quantify that? for example what's STRM 95% confidence accuracy? [I think this needs a probabilistic approach, see my other reply to this thread.] I am not sure it's worth spending too much time on (it might be for very long distance visibility -- I just don't know). it would be good for us to do a rough back of the envelope calc to justify that before fully forgetting about it. I guess for the worst case scenario we could try the views from Mt. Everest and/or Olympus Mons and see what difference it makes. -- that would rock :) Ben thanks, Hamish -- Files attached to this email may be in ISO 26300 format (OASIS Open Document Format). If you have difficulty opening them, please visit http://iso26300.info for more information. ___ grass-user mailing list grass-user@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-user
Re: [GRASS-user] GRASS earth curvature correction (viewsheds, LOS)
- Original Message - On Thu, May 26, 2011 at 12:26 PM, Benjamin Ducke benjamin.du...@oxfordarch.co.uk wrote: Hm-hm. Citing from the website: The problem is that the ratio of change due to air to curvature is not 1:7 (0.13), as the standard refraction coefficient suggests. It is 0.325. As far as I can tell, this is a mis-understanding. The value 0.325 applies to radio waves. Visible light is very close to 1:7. What if I am interested in radio waves, not visible light, e.g. for antenna relay positions? IMHO, that refraction coefficient should not be hard-coded. Agreed. It's a settable value in r.ecurv.comp and should also be one in all GRASS modules that have refraction compensation. Ben I realize the whole discourse is somewhat clouded. But I don't have access to most of the relevant literature for the time being, nor do I have the necessary scientific background Me neither. But any correction should take into account that the observer is not necessarily a human without optical equipment (telescope etc), but can also be some technical device, e.g. a sender or receiver of whatever signals. my .2c Markus M -- Files attached to this email may be in ISO 26300 format (OASIS Open Document Format). If you have difficulty opening them, please visit http://iso26300.info for more information. ___ grass-user mailing list grass-user@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-user
Re: [GRASS-user] GRASS earth curvature correction (viewsheds, LOS)
As far as I can tell, this is a mis-understanding. The value 0.325 applies to radio waves. Visible light is very close to 1:7. What if I am interested in radio waves, not visible light, e.g. for antenna relay positions? IMHO, that refraction coefficient should not be hard-coded. Agreed. It's a settable value in r.ecurv.comp and should also be one in all GRASS modules that have refraction compensation. ok good point, wavelength matters VHF LOS propagation distance is an important problem to model. r.viewshed now has a user settable refection coefficient in the range 0.0-1.0. coeff=0.0 is like no refraction, 1.0 totally cancels out any curvature of the Earth. btw 1/7 is not 0.13, it's ~0.143. (to be picky) Hamish ___ grass-user mailing list grass-user@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-user
Re: [GRASS-user] GRASS earth curvature correction (viewsheds, LOS)
Benjamin wrote: I just recently had the opportunity to revisit the issue of earth curvature correction in r.los (and, by extension, r.cva). The curvature of the Earth's surface means that visibility will be over-estimated if the line-of-sight algorithm does not correct for its effect. I initially implemented a tentative correction in r.cva and it was with some anxiety that I saw that (untested) method spread into the source code of r.los. However, after some testing and comparison with the output of ArcGIS' Viewshed module, I can now confirm that the correction method works just fine. It gives confidence sure, but there's no way of knowing if their black box is actually correct or not, maybe we make the same mistakes? :) All the better to follow/cite published articles. So that's the good news. The remaining itch is that atmospheric refraction is not corrected for in any GRASS line-of-sight based module. That effect is smaller, but it works against the effect of earth curvature. This means that a module which corrects for curvature, but not refraction, will tend to over-compensate (by about 6/7). I have not tested r.viewshed's curvature correction. But if it uses the same method as r.los, then the same observations apply. I'm not sure, but I suspect it might. MarkusM? As an alternative to all this mess, I have attached a little script which allows the user to pre-process the digital elevation model, in effect producing a fake planar model that fools LOS algorithms into operating in a plane instead of on a curved surface. This little trick works as long as the region is not more than a few hundred kilometers across. Kudos to Bill Huber of Quantitative Decisions, who made the details of the computations available via the user forums of a well-known GIS producer. More details can be found in the HTML man page included in the attached ZIP file. Pre-processing the elevation data and then turning OFF any built-in correction methods of any LOS module used will make the computations transparent and more comparable. certainly the ideal solution is to have both curvature and refection corrections implemented as flags in the new improved r.viewshed. (say, is that now ready for moving into grass7? *) Does refraction only work in the vertical or would you be able to slightly see around some distant volcano horizontally? ..perhaps the phenomenon is related to the atm pressure gradient, in which case purely a vertical-only effect..? [*] @MarkusM: re. changing out-of-view to NULL in r.viewshed, I now notice in the help page that NULL was previously reserved for NULLs in the original DEM. But in that case I suspect that everything in the shadow of the DEM's NULLs should also be NULL, as the visibility will be unknowable. I'm mildly in favour of keeping the code as-is, and changing the help page to match. Btw.: When comparing the output of ArcGIS' Viewshed with that of r.los (or any other LOS modules I have tried, including some from SEXTANTE), you will notice that the ArcGIS viewsheds are always more contiguous and less noisy. That software seems to use some internal tolerance or smoothing to achieve this effect. Do you think that noise happens because of a translation of the coarse horizontal grid cell size when it is translated into the vertical plane? I am not sure it's a desirable thing to emulate, though. Better smooth the result manually and in a controlled manner using e.g. a median filter, if desired. absolutely. If it's a common task maybe offer a flag to run that automatically, but certainly don't mess with the results by default. thanks, Hamish ___ grass-user mailing list grass-user@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-user
Re: [GRASS-user] GRASS earth curvature correction (viewsheds, LOS)
On Wed, May 25, 2011 at 12:34 PM, Hamish hamis...@yahoo.com wrote: Benjamin wrote: [...] I have not tested r.viewshed's curvature correction. But if it uses the same method as r.los, then the same observations apply. I'm not sure, but I suspect it might. MarkusM? r.viewshed's correction for earth curvature is copied as is from (recent) r.los. [...] [*] @MarkusM: re. changing out-of-view to NULL in r.viewshed, I now notice in the help page that NULL was previously reserved for NULLs in the original DEM. But in that case I suspect that everything in the shadow of the DEM's NULLs should also be NULL, as the visibility will be unknowable. I'm mildly in favour of keeping the code as-is, and changing the help page to match. The original behaviour of r.viewshed was to distinguish between visible and invisible cells and cells for which visibility could not be determined because input elevation is NULL. r.los on the contrary sets both invisible cells and cells for which visibility could not be determined to NULL. I synchronized r.viewshed to r.los because r.los is in the main branches whereas r.viewshed is in addons. Distinguishing between invisible cells and cells for which visibility could not be determined might provide useful information, though. Markus M ___ grass-user mailing list grass-user@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-user
Re: [GRASS-user] GRASS earth curvature correction (viewsheds, LOS)
[snip] However, after some testing and comparison with the output of ArcGIS' Viewshed module, I can now confirm that the correction method works just fine. It gives confidence sure, but there's no way of knowing if their black box is actually correct or not, maybe we make the same mistakes? :) All the better to follow/cite published articles. Of course, that will always be a problem. But I have compared three independent methods: r.los with built-in correction, ArcGIS with built-in correction, and r.ecurv.comp. In general, the outputs are very similar, even down to small detail. The remaining differences must be in implementation details and the fact that r.los does not implement atmospheric refraction. I have also run some of the LOS algorithms in SEXANTE (on a DEM that was processed with r.ecurv.comp, since they lack any built-in correction). Again, results were very much the same as r.los and ArcGIS. Only ArcGIS outputs those contiguous viewsheds, though. So they must have done something to tune the output. certainly the ideal solution is to have both curvature and refection corrections implemented as flags in the new improved r.viewshed. (say, is that now ready for moving into grass7? *) That should be really easy to do. All that's needed is to amend the existing correction but taking away 1/7 to account for the adverse effect of refraction. Does refraction only work in the vertical or would you be able to slightly see around some distant volcano horizontally? ..perhaps the phenomenon is related to the atm pressure gradient, in which case purely a vertical-only effect..? Well, that refraction correction is really a rough simplification of reality. Essentially, it uses the same amount of correction as ArcGIS. There is some justification for this. You can find links to articles here: http://mapaspects.org/content/effects-curvature-earth-refraction-light-air-and-fuzzy-viewsheds-arcgis-92 (@Markus N: maybe that answers your question, as well?) But in summary, accounting for realistic refraction conditions would be much more complex, as it would also have to take into plus different refraction at different elevations, etc. But given that most DEMs have an inherent vertical error that is greater than the influence of these effects, I am not sure it's worth spending too much time on (it might be for very long distance visibility -- I just don't know). Do you think that noise happens because of a translation of the coarse horizontal grid cell size when it is translated into the vertical plane? That's a really interesting thought. It could very well be a quantization effect of that kind. That might explain why the noisy patches seem to show structural features of the DEM. Cheers, Ben -- Files attached to this email may be in ISO 26300 format (OASIS Open Document Format). If you have difficulty opening them, please visit http://iso26300.info for more information. ___ grass-user mailing list grass-user@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-user
Re: [GRASS-user] GRASS earth curvature correction (viewsheds, LOS)
Markus Metz wrote: The original behaviour of r.viewshed was to distinguish between visible and invisible cells and cells for which visibility could not be determined because input elevation is NULL. r.los on the contrary sets both invisible cells and cells for which visibility could not be determined to NULL. I synchronized r.viewshed to r.los because r.los is in the main branches whereas r.viewshed is in addons. Distinguishing between invisible cells and cells for which visibility could not be determined might provide useful information, though. well, AFAIR r.los uses NULL for that simply because that's what I decided to do when I added NULL support to the module some years ago. So to distinguish between uncalculated (because beyond the requested search radius?), and in-search-region but non-visible? Well, it's not to me, but I guess potentially that's useful to someone somewhere. I hate to needlessly lose information, but in general I think NULL for non-visible is what you'd want the output to show most of the time. So again I'm mildly in favour of keeping out-of-sight = NULL, but not of too strong an opinion about it. Hamish ___ grass-user mailing list grass-user@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-user
