Re: [GRASS-user] creating high-resolution DMT

2017-04-15 Thread Benjamin Ducke 
On 15/04/17 05:31, Vaclav Petras wrote:
> Back to filling holes in lidar data, I also think that the "fill in the
> gaps" approach is quite promising, so I ported the r.fill.gaps module to G7:

Great! I have read the diff and that was very
helpful for me. I can see that the old and
new raster APIs have only minimal differences.
Porting seems very straight-forward.

I hope that r.fill.gaps will be useful for others.
I did not have LiDAR in mind when I wrote it,
but now it seems so obvious that LiDAR processing
is an important domain of application for the module.

Cheers!

Ben

> 
> https://grass.osgeo.org/grass72/manuals/addons/r.fill.gaps.html
> 
> I didn't review the code, but the documentation is very nice and
> detailed. I did some mostly formatting updates for the current
> submitting guidelines and added complete example with lidar data.
> 
> On Tue, Mar 21, 2017 at 7:29 AM, Benjamin Ducke  > wrote:
> 
> 
> I just realized that I have never committed "r.fill.gaps"
> to the add-ins repo.
> 
> 
> Really nice module, please, find more of these in your shelf :-)
> 
> Thanks!
> Vaclav
> 
>  
> 
> I have just done that now:
> 
> http://grasswiki.osgeo.org/wiki/AddOns/GRASS_6#r.fill.gaps
> 
> 
> Its purpose is filling small gaps in otherwise dense
> data using IDW with a pre-computed weights matrix.
> It works on rasterized data, so you simply use v.to.rast
> on your vector points first. Make sure to set the
> cell size small enough so that you don't get many
> vector points falling into the same cell. The result
> will be an oversampled raster with a lot of small
> "no data" cells. That's exactly the intended input for
> r.fill.gaps!
> 
> This is not multi-core code, (parallelizing
> interpolation algorithms is hard, because you need to
> segment the data and then you need to deal with the
> seams between the segments), but it is very, very fast,
> as long as you keep the IDW radius small.
> 
> The code is optimized to death, which makes it very
> hard to read. Another drawback is that it was written
> for GRASS 6 (but converting it to GRASS 7 should not
> be hard, since it uses only the basic raster API).
> 
> r.fill.gaps is not useful for filling large gaps, both
> in terms of performance (for large IDW radii) and
> interpolation quality (it's IDW -- enough said).
> 
> 
> 



-- 
Dr. Benjamin Ducke
{*} Geospatial Consultant
{*} GIS Developer

Spatial technology for the masses, not the classes:
experience free and open source GIS at http://gvsigce.org
___
grass-user mailing list
grass-user@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] creating high-resolution DMT

2017-04-14 Thread Vaclav Petras 
On Tue, Mar 21, 2017 at 10:01 AM, Markus Neteler  wrote:
>
> > Another drawback is that it was written
> > for GRASS 6 (but converting it to GRASS 7 should not
> > be hard, since it uses only the basic raster API).
>
> FWIW New names can be looked up here:
> https://trac.osgeo.org/grass/wiki/Grass7/RasterLib/ListOfFunctions

For those interested in what was required for the port from 6 to 7:

https://trac.osgeo.org/grass/changeset/70880
___
grass-user mailing list
grass-user@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] creating high-resolution DMT

2017-04-14 Thread Vaclav Petras 
Back to filling holes in lidar data, I also think that the "fill in the
gaps" approach is quite promising, so I ported the r.fill.gaps module to G7:

https://grass.osgeo.org/grass72/manuals/addons/r.fill.gaps.html

I didn't review the code, but the documentation is very nice and detailed.
I did some mostly formatting updates for the current submitting guidelines
and added complete example with lidar data.

On Tue, Mar 21, 2017 at 7:29 AM, Benjamin Ducke 
wrote:

>
> I just realized that I have never committed "r.fill.gaps"
> to the add-ins repo.
>
>
Really nice module, please, find more of these in your shelf :-)

Thanks!
Vaclav



> I have just done that now:
>
> http://grasswiki.osgeo.org/wiki/AddOns/GRASS_6#r.fill.gaps
>
> Its purpose is filling small gaps in otherwise dense
> data using IDW with a pre-computed weights matrix.
> It works on rasterized data, so you simply use v.to.rast
> on your vector points first. Make sure to set the
> cell size small enough so that you don't get many
> vector points falling into the same cell. The result
> will be an oversampled raster with a lot of small
> "no data" cells. That's exactly the intended input for
> r.fill.gaps!
>
> This is not multi-core code, (parallelizing
> interpolation algorithms is hard, because you need to
> segment the data and then you need to deal with the
> seams between the segments), but it is very, very fast,
> as long as you keep the IDW radius small.
>
> The code is optimized to death, which makes it very
> hard to read. Another drawback is that it was written
> for GRASS 6 (but converting it to GRASS 7 should not
> be hard, since it uses only the basic raster API).
>
> r.fill.gaps is not useful for filling large gaps, both
> in terms of performance (for large IDW radii) and
> interpolation quality (it's IDW -- enough said).
>
___
grass-user mailing list
grass-user@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] creating high-resolution DMT

2017-03-21 Thread Markus Neteler 
On Tue, Mar 21, 2017 at 12:29 PM, Benjamin Ducke  wrote:
...
> I just realized that I have never committed "r.fill.gaps"
> to the add-ins repo.
>
> I have just done that now:
> http://grasswiki.osgeo.org/wiki/AddOns/GRASS_6#r.fill.gaps

cool!

...
> The code is optimized to death, which makes it very
> hard to read. Another drawback is that it was written
> for GRASS 6 (but converting it to GRASS 7 should not
> be hard, since it uses only the basic raster API).

FWIW New names can be looked up here:
https://trac.osgeo.org/grass/wiki/Grass7/RasterLib/ListOfFunctions

Best
Markus
___
grass-user mailing list
grass-user@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] creating high-resolution DMT

2017-03-21 Thread Benjamin Ducke 
On 21/03/17 10:12, Blumentrath, Stefan wrote:
> Hei Martin,
> 
> Same experience here. For larger datasets from LiDAR I rather use IDW
> for interpolation.

I just realized that I have never committed "r.fill.gaps"
to the add-ins repo.

I have just done that now:

http://grasswiki.osgeo.org/wiki/AddOns/GRASS_6#r.fill.gaps

Its purpose is filling small gaps in otherwise dense
data using IDW with a pre-computed weights matrix.
It works on rasterized data, so you simply use v.to.rast
on your vector points first. Make sure to set the
cell size small enough so that you don't get many
vector points falling into the same cell. The result
will be an oversampled raster with a lot of small
"no data" cells. That's exactly the intended input for
r.fill.gaps!

This is not multi-core code, (parallelizing
interpolation algorithms is hard, because you need to
segment the data and then you need to deal with the
seams between the segments), but it is very, very fast,
as long as you keep the IDW radius small.

The code is optimized to death, which makes it very
hard to read. Another drawback is that it was written
for GRASS 6 (but converting it to GRASS 7 should not
be hard, since it uses only the basic raster API).

r.fill.gaps is not useful for filling large gaps, both
in terms of performance (for large IDW radii) and
interpolation quality (it's IDW -- enough said).

Best,

Ben

> 
> The "per hole filling" in r.fillnulls is not utilizing multiple cores
> and if you have a lot of holes that can slow down the whole process
> significantly. Not sure if a mask can help to avoid filling no data
> areas around your data.
> 
> For r.fillnulls there is an enhancement ticket (with a patch) for
> speedup of the patching of the maps at the end... 
> https://trac.osgeo.org/grass/ticket/1938 However, it does not tackle
> parallel filling of holes...
> 
> Cheers Stefan
> 
> ___ grass-user mailing
> list grass-user@lists.osgeo.org 
> https://lists.osgeo.org/mailman/listinfo/grass-user
> 



-- 
Dr. Benjamin Ducke
{*} Geospatial Consultant
{*} GIS Developer

Spatial technology for the masses, not the classes:
experience free and open source GIS at http://gvsigce.org
___
grass-user mailing list
grass-user@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] creating high-resolution DMT

2017-03-21 Thread Blumentrath, Stefan 
Hei Martin,

Same experience here.
For larger datasets from LiDAR I rather use IDW for interpolation.

The "per hole filling" in r.fillnulls is not utilizing multiple cores and if 
you have a lot of holes that can slow down the whole process significantly.
Not sure if a mask can help to avoid filling no data areas around your data.

For r.fillnulls there is an enhancement ticket (with a patch) for speedup of 
the patching of the maps at the end...
https://trac.osgeo.org/grass/ticket/1938
However, it does not tackle parallel filling of holes...

Cheers
Stefan 

___
grass-user mailing list
grass-user@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-user

[GRASS-user] creating high-resolution DMT

2017-03-21 Thread Martin Landa 
Hi,

recently I was playing with DMT products available in Czech Republic, basically:

* DMR4G regular grid of points (spacing 5m)
* DMR5G irregular point network (average spacing ~3m)

When creating raster-based DMT I tested scenarios bellow:

1) create raster with resolution 5m from DMR4G (no interpolation, just
g.region + r.in.xyz/r.in.lidar)

2) create raster with high resolution (less than 3m) with v.surf.rst
(default parameters defined)

Unfortunately v.surf.rst is quite slow (tile with 5e5 points and
region 2000x2500 cells took on my pc about 3hours). I enjoyed very
much parallelization support in GRASS 7.3. Using 8 cores I am able to
process one tile in 20min. It's great. Probably also some tunning
parameters could help(?)

I was thinking about other scenarios as:

3) import points using r.in.xyz/r.in.lidar on desired resolution and
fill holes with r.fillnuls (but it will be also slow)

4) use v.surf.bspline

What is your experience and how you would process such data?

Thanks for your feedback in advance! Martin

-- 
Martin Landa
http://geo.fsv.cvut.cz/gwiki/Landa
http://gismentors.cz/mentors/landa
___
grass-user mailing list
grass-user@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-user