Re: [GRASS-user] zonal statistics/metrics - beyond simple statistics

[Bernardo Santos]

 Hi Moritz,
thanks for the comments.
I didn't know that r.mask had a 'cat' option, this makes the whole process more 
direct. The same for using g.region raster=MASK.
Also, the hint about keeping info as text (as you did in i.segment.stats) and 
raster is a very good idea to make the process faster. I'll implement those 
things to speed up the code.
I don't know if it makes it clearer, but what we wanted is to calculate several 
metrics and statistics, using Brazilian city limits as zones (a total of ~5,600 
cities), based on land use maps for the whole country (resolution = 30m), for 
all years from 2000 to 2016. Also, we didn't wanted to calculate only 
general/basic statistics such as average, sum, etc., but also more specific 
metrics such as the area and percentage of different cover classes, edge 
amount, number of forest patches, isolation between forest patches and 
others.That's why I chose to build it in a way we write a given function (that 
calculated any of these metrics) and then we use it as an argument for the 
process, so that we loop over polygons and over input rasters to calculate 
these metrics for each city.
That was my initial aim, but I guess this may also be useful in other cases.
I know r.li (but didn't tested r.pi yet), the metrics it calculates are very 
interesting but somehow limited (of course I could write other ones based on 
the r.li.daemon core function, but I am not very well versed in C to use it 
yet...). Even though, I agree I could use maybe a combination of r.li and make 
some calculations with the output info of r.univar and r.stats to get the same 
numbers, and maybe I would be able to avoid loops.I'll think about it.
Thanks, Moritz and Stephan!BernardoEm quinta-feira, 12 de abril de 2018 
13:15:17 BRT, Moritz Lennert  escreveu:  
 
 On 12/04/18 16:56, Bernardo Santos wrote:
> For example, I am still not sure how I would use a combination of 
> r.univar/r.stats and some kind of input raster map to produce, say, 
> zonal info of number of habitat patches, structural or functional 
> connectivity, isolation, or other more complex metrics used in landscape 
> ecology.
> That's way I was building something like that, so that we can just write 
> an alternative function using any raster input map that returns the 
> value of one of such metrics, and then use it to calculate them over 
> sets of zones.

Have you looked into the landscape metrics modules, notably the r.li 
suite of modules, but also the r.pi suite in the addons (although ISTR 
that the latter is not really usable in its current state) ?

Moritz
  ___
grass-user mailing list
grass-user@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] zonal statistics/metrics - beyond simple statistics

[Bernardo Santos]

 Hi Stephan,
nice points you raised. I didn't know about the Unix philosophy, and indeed it 
seems as a more solid / simple way of building things...
And yeah, I agree that these metrics/statistics I mentioned are not 'general' 
at all.What I meant by general is that we can write any other function that has 
one or more raster/vector maps as input, makes some calculation and returns a 
value, and just use this function as input for this 'GeneralizedZonalStats', 
without having to change anything else...
But yes, I agree that calling that 'Generalized' or 'General' may not be the 
best option...
Best,Bernardo
Em quinta-feira, 12 de abril de 2018 12:18:20 BRT, Stefan Blumentrath 
<stefan.blumentr...@nina.no> escreveu:  
 
 #yiv8564277711 #yiv8564277711 -- _filtered #yiv8564277711 
{font-family:Helvetica;panose-1:2 11 6 4 2 2 2 2 2 4;} _filtered #yiv8564277711 
{panose-1:2 4 5 3 5 4 6 3 2 4;} _filtered #yiv8564277711 
{font-family:Calibri;panose-1:2 15 5 2 2 2 4 3 2 4;} _filtered #yiv8564277711 
{panose-1:0 0 0 0 0 0 0 0 0 0;}#yiv8564277711 #yiv8564277711 
p.yiv8564277711MsoNormal, #yiv8564277711 li.yiv8564277711MsoNormal, 
#yiv8564277711 div.yiv8564277711MsoNormal 
{margin:0cm;margin-bottom:.0001pt;font-size:12.0pt;font-family:New 
serif;}#yiv8564277711 a:link, #yiv8564277711 span.yiv8564277711MsoHyperlink 
{color:blue;text-decoration:underline;}#yiv8564277711 a:visited, #yiv8564277711 
span.yiv8564277711MsoHyperlinkFollowed 
{color:purple;text-decoration:underline;}#yiv8564277711 
p.yiv8564277711msonormal0, #yiv8564277711 li.yiv8564277711msonormal0, 
#yiv8564277711 div.yiv8564277711msonormal0 
{margin-right:0cm;margin-left:0cm;font-size:12.0pt;font-family:New 
serif;}#yiv8564277711 p.yiv8564277711msonormal0, #yiv8564277711 
li.yiv8564277711msonormal0, #yiv8564277711 div.yiv8564277711msonormal0 
{margin-right:0cm;margin-left:0cm;font-size:12.0pt;font-family:New 
serif;}#yiv8564277711 p.yiv8564277711msonormal, #yiv8564277711 
li.yiv8564277711msonormal, #yiv8564277711 div.yiv8564277711msonormal 
{margin-right:0cm;margin-left:0cm;font-size:12.0pt;font-family:New 
serif;}#yiv8564277711 p.yiv8564277711msochpdefault, #yiv8564277711 
li.yiv8564277711msochpdefault, #yiv8564277711 div.yiv8564277711msochpdefault 
{margin-right:0cm;margin-left:0cm;font-size:12.0pt;font-family:New 
serif;}#yiv8564277711 span.yiv8564277711msohyperlink {}#yiv8564277711 
span.yiv8564277711msohyperlinkfollowed {}#yiv8564277711 
span.yiv8564277711emailstyle18 {}#yiv8564277711 p.yiv8564277711msonormal1, 
#yiv8564277711 li.yiv8564277711msonormal1, #yiv8564277711 
div.yiv8564277711msonormal1 
{margin:0cm;margin-bottom:.0001pt;font-size:12.0pt;font-family:serif;}#yiv8564277711
 span.yiv8564277711msohyperlink1 
{color:blue;text-decoration:underline;}#yiv8564277711 
span.yiv8564277711msohyperlinkfollowed1 
{color:purple;text-decoration:underline;}#yiv8564277711 
p.yiv8564277711msonormal01, #yiv8564277711 li.yiv8564277711msonormal01, 
#yiv8564277711 div.yiv8564277711msonormal01 
{margin-right:0cm;margin-left:0cm;font-size:12.0pt;font-family:serif;}#yiv8564277711
 span.yiv8564277711emailstyle181 
{font-family:sans-serif;color:#1F497D;}#yiv8564277711 
p.yiv8564277711msochpdefault1, #yiv8564277711 li.yiv8564277711msochpdefault1, 
#yiv8564277711 div.yiv8564277711msochpdefault1 
{margin-right:0cm;margin-left:0cm;font-size:10.0pt;font-family:New 
serif;}#yiv8564277711 span.yiv8564277711EmailStyle30 
{font-family:sans-serif;color:#1F497D;}#yiv8564277711 
.yiv8564277711MsoChpDefault {font-size:10.0pt;} _filtered #yiv8564277711 
{margin:70.85pt 70.85pt 70.85pt 70.85pt;}#yiv8564277711 
div.yiv8564277711WordSection1 {}#yiv8564277711 
Hi Bernado,
 
  
 
It depends what you consider general. Even if I work with landscape ecology 
myself, I would assume landscape metrics or functional connectivity to be 
rather “special” than “general” statistics.
 
  
 
Putting a lot of functionality into one module can be very useful for some 
cases, but can come with a number of drawbacks as well.
 
  
 
A general Unix concept is to do one thing and do it well, and rather write 
modular programs that work together than very complex single tools…
 
  
 
Cheers
 
Stefan
 
  
 
1:https://en.wikipedia.org/wiki/Unix_philosophy
 
  
 
From: Bernardo Santos <bernardo_brand...@yahoo.com.br>
Sent: torsdag 12. april 2018 16.57
To: Helmut Kudrnovsky <hel...@web.de>; grass-user@lists.osgeo.org; Stefan 
Blumentrath <stefan.blumentr...@nina.no>
Subject: Re: RE: [GRASS-user] zonal statistics/metrics - beyond simple 
statistics
 
  
 
Hi Stephan,
 
  
 
Some minutes before you sent me this e-mails, I'd just found out about 
v.db.select -r. It worked and made the process much faster. Thanks for that 
anyway.
 
  
 
In my case, the polygons do not overlap. Indeed, many things could be done with 
r.univar, r.stats or a combination of them and their output info. However, my 
aim was to think about a way of generalizing it, so that we would be able to 
use m

Re: [GRASS-user] zonal statistics/metrics - beyond simple statistics

[Moritz Lennert]

On 12/04/18 16:56, Bernardo Santos wrote:
For example, I am still not sure how I would use a combination of 
r.univar/r.stats and some kind of input raster map to produce, say, 
zonal info of number of habitat patches, structural or functional 
connectivity, isolation, or other more complex metrics used in landscape 
ecology.
That's way I was building something like that, so that we can just write 
an alternative function using any raster input map that returns the 
value of one of such metrics, and then use it to calculate them over 
sets of zones.


Have you looked into the landscape metrics modules, notably the r.li 
suite of modules, but also the r.pi suite in the addons (although ISTR 
that the latter is not really usable in its current state) ?


Moritz
___
grass-user mailing list
grass-user@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] zonal statistics/metrics - beyond simple statistics

[Stefan Blumentrath]

Hi Bernado,

It depends what you consider general. Even if I work with landscape ecology 
myself, I would assume landscape metrics or functional connectivity to be 
rather “special” than “general” statistics.

Putting a lot of functionality into one module can be very useful for some 
cases, but can come with a number of drawbacks as well.

A general Unix concept is to do one thing and do it well, and rather write 
modular programs that work together than very complex single tools…

Cheers
Stefan

1: https://en.wikipedia.org/wiki/Unix_philosophy

From: Bernardo Santos <bernardo_brand...@yahoo.com.br>
Sent: torsdag 12. april 2018 16.57
To: Helmut Kudrnovsky <hel...@web.de>; grass-user@lists.osgeo.org; Stefan 
Blumentrath <stefan.blumentr...@nina.no>
Subject: Re: RE: [GRASS-user] zonal statistics/metrics - beyond simple 
statistics

Hi Stephan,

Some minutes before you sent me this e-mails, I'd just found out about 
v.db.select -r. It worked and made the process much faster. Thanks for that 
anyway.

In my case, the polygons do not overlap. Indeed, many things could be done with 
r.univar, r.stats or a combination of them and their output info. However, my 
aim was to think about a way of generalizing it, so that we would be able to 
use many other functions from the raster maps using the same architecture for 
calculating the zonal statistics/metrics.

For example, I am still not sure how I would use a combination of 
r.univar/r.stats and some kind of input raster map to produce, say, zonal info 
of number of habitat patches, structural or functional connectivity, isolation, 
or other more complex metrics used in landscape ecology.
That's way I was building something like that, so that we can just write an 
alternative function using any raster input map that returns the value of one 
of such metrics, and then use it to calculate them over sets of zones.

Best,
B

PS: thanks for sharing your code! This v.rast.bufferstats seems very useful, 
and I can think of some very interesting applications of that. I helped a 
friend to write something similar but using ArcPy as a ArcMap toolbox, and we 
applied it to some interesting cases in the delimitation of Protected Areas' 
buffer zones (but still did not published it). But doing that with a free and 
open source tool would be much more interesting!
Em quarta-feira, 11 de abril de 2018 18:32:18 BRT, Stefan Blumentrath 
<stefan.blumentr...@nina.no<mailto:stefan.blumentr...@nina.no>> escreveu:



Hi Bernado,



Please find attached the «work-in-progress» version of v.rast.bufferstats for 
some more inspiration.



v.db.select -r does what you are looking for:

https://grass.osgeo.org/grass74/manuals/v.db.select.html



However, if the polygons you are working with don`t overlap, looping should not 
be required, and you should be able to use r.univar with zones or r.stats for 
all polygons at once…



Cheers

Stefan







From: Bernardo Santos 
<bernardo_brand...@yahoo.com.br<mailto:bernardo_brand...@yahoo.com.br>>
Sent: onsdag 11. april 2018 20.59
To: Helmut Kudrnovsky <hel...@web.de<mailto:hel...@web.de>>; 
grass-user@lists.osgeo.org<mailto:grass-user@lists.osgeo.org>; Stefan 
Blumentrath <stefan.blumentr...@nina.no<mailto:stefan.blumentr...@nina.no>>
Subject: Re: [GRASS-user] zonal statistics/metrics - beyond simple statistics



Hi Helmut and Stefan,



thanks for sharing that with me, it is good to know that there are similar 
ongoing approaches to solve related issues.



Helmut,

very nice addon. However, it seems to me that the part on calculating 
statistics is still based on v.rast.stats, am I right?



Stefan, the question you pose is indeed very related. I took a better look at 
my code and I noticed that what really takes time is not the process of making 
a mask, but what comes before: the rasterization of one of the polygons of the 
input vector.

What I do is:

1) g.region using the vector layer, aligning it to the first of the input 
rasters

2) v.to.rast selecting only one of the polygons at a time from the vector layer

3) r.mask using the rasterized polygon

4) g.region zoom=rasterized_polygon

5) calculate any function for all raster maps (in principle using a loop), 
using this small region and the mask, and attaching the results to the 
attribute table

6) repeat 1-5 for all polygons in the input vector



However, I noticed now that what takes time is v.rast.stats, since it depends 
on the whole region selected, which in my case is much bigger than a single 
polygon (the vector is a set of ~6,000 small polygons).

Do you guys know if there is a way of using a single polygon from a vector to 
define the region using something like g.region?

[this would be really great!!]

something like

g.region vector=vector_of_interest cat=1



Best,

Bernardo

Em terça-feira, 10 de abril de 2018 06:01:34 BRT, Stefan Blumentrath 
<stefan.blumentr...@nina.no<mailto:stefan

Re: [GRASS-user] zonal statistics/metrics - beyond simple statistics

[Bernardo Santos]

 Hi Stephan,

Some minutes before you sent me this e-mails, I'd just found out about 
v.db.select -r. It worked and made the process much faster. Thanks for that 
anyway.
In my case, the polygons do not overlap. Indeed, many things could be done with 
r.univar, r.stats or a combination of them and their output info. However, my 
aim was to think about a way of generalizing it, so that we would be able to 
use many other functions from the raster maps using the same architecture for 
calculating the zonal statistics/metrics.
For example, I am still not sure how I would use a combination of 
r.univar/r.stats and some kind of input raster map to produce, say, zonal info 
of number of habitat patches, structural or functional connectivity, isolation, 
or other more complex metrics used in landscape ecology.That's way I was 
building something like that, so that we can just write an alternative function 
using any raster input map that returns the value of one of such metrics, and 
then use it to calculate them over sets of zones.
Best,B
PS: thanks for sharing your code! This v.rast.bufferstats seems very useful, 
and I can think of some very interesting applications of that. I helped a 
friend to write something similar but using ArcPy as a ArcMap toolbox, and we 
applied it to some interesting cases in the delimitation of Protected Areas' 
buffer zones (but still did not published it). But doing that with a free and 
open source tool would be much more interesting! Em quarta-feira, 11 de 
abril de 2018 18:32:18 BRT, Stefan Blumentrath <stefan.blumentr...@nina.no> 
escreveu:  
 
 #yiv6359193583 #yiv6359193583 -- _filtered #yiv6359193583 
{font-family:Helvetica;panose-1:2 11 6 4 2 2 2 2 2 4;} _filtered #yiv6359193583 
{panose-1:2 4 5 3 5 4 6 3 2 4;} _filtered #yiv6359193583 
{font-family:Calibri;panose-1:2 15 5 2 2 2 4 3 2 4;}#yiv6359193583 
#yiv6359193583 p.yiv6359193583MsoNormal, #yiv6359193583 
li.yiv6359193583MsoNormal, #yiv6359193583 div.yiv6359193583MsoNormal 
{margin:0cm;margin-bottom:.0001pt;font-size:12.0pt;font-family:New 
serif;}#yiv6359193583 a:link, #yiv6359193583 span.yiv6359193583MsoHyperlink 
{color:blue;text-decoration:underline;}#yiv6359193583 a:visited, #yiv6359193583 
span.yiv6359193583MsoHyperlinkFollowed 
{color:purple;text-decoration:underline;}#yiv6359193583 
p.yiv6359193583msonormal0, #yiv6359193583 li.yiv6359193583msonormal0, 
#yiv6359193583 div.yiv6359193583msonormal0 
{margin-right:0cm;margin-left:0cm;font-size:12.0pt;font-family:New 
serif;}#yiv6359193583 span.yiv6359193583EmailStyle18 
{font-family:sans-serif;color:#1F497D;}#yiv6359193583 
.yiv6359193583MsoChpDefault {font-size:10.0pt;} _filtered #yiv6359193583 
{margin:70.85pt 70.85pt 70.85pt 70.85pt;}#yiv6359193583 
div.yiv6359193583WordSection1 {}#yiv6359193583 
Hi Bernado,
 
  
 
Please find attached the «work-in-progress» version of v.rast.bufferstats for 
some more inspiration.
 
  
 
v.db.select -r does what you are looking for:
 
https://grass.osgeo.org/grass74/manuals/v.db.select.html
 
  
 
However, if the polygons you are working with don`t overlap, looping should not 
be required, and you should be able to use r.univar with zones or r.stats for 
all polygons at once…
 
  
 
Cheers
 
Stefan
 
  
 
  
 
  
 
From: Bernardo Santos <bernardo_brand...@yahoo.com.br>
Sent: onsdag 11. april 2018 20.59
To: Helmut Kudrnovsky <hel...@web.de>; grass-user@lists.osgeo.org; Stefan 
Blumentrath <stefan.blumentr...@nina.no>
Subject: Re: [GRASS-user] zonal statistics/metrics - beyond simple statistics
 
  
 
Hi Helmut and Stefan,
 
  
 
thanks for sharing that with me, it is good to know that there are 
similarongoing approaches to solve related issues.
 
  
 
Helmut,
 
very nice addon. However, it seems to me that the part on calculating 
statistics is still based on v.rast.stats, am I right?
 
  
 
Stefan, the question you pose is indeed very related. I took a better look at 
my code and I noticed that what really takes time is not the process of making 
a mask, but what comes before: the rasterization of one of the polygons of the 
input vector.
 
What I do is:
 
1) g.region using the vector layer, aligning it to the first of the input 
rasters
 
2) v.to.rast selecting only one of the polygons at a time from the vector layer
 
3) r.mask using the rasterized polygon
 
4) g.region zoom=rasterized_polygon
 
5) calculate any function for all raster maps (in principle using a loop), 
using this small region and the mask, and attaching the results to the 
attribute table
 
6) repeat 1-5 for all polygons in the input vector
 
  
 
However, I noticed now that what takes time is v.rast.stats, since it depends 
on the whole region selected, which in my case is much bigger than a single 
polygon (the vector is a set of ~6,000 small polygons).
 
Do you guys know if there is a way of using a single polygon from a vector to 
define the region using something like g.region?
 
[this would be really great!!]
 
somethin

Re: [GRASS-user] zonal statistics/metrics - beyond simple statistics

[Moritz Lennert]

On 11/04/18 20:58, Bernardo Santos wrote:

Hi Helmut and Stefan,

thanks for sharing that with me, it is good to know that there are 
similar ongoing approaches to solve related issues.


Helmut,
very nice addon. However, it seems to me that the part on calculating 
statistics is still based on v.rast.stats, am I right?


Stefan, the question you pose is indeed very related. I took a better 
look at my code and I noticed that what really takes time is not the 
process of making a mask, but what comes before: the rasterization of 
one of the polygons of the input vector.

What I do is:
1) g.region using the vector layer, aligning it to the first of the 
input rasters
2) v.to.rast selecting only one of the polygons at a time from the 
vector layer

3) r.mask using the rasterized polygon


You can actually use r.mask directly with the vector map using the 
vector and cat/where parameters.



4) g.region zoom=rasterized_polygon
5) calculate any function for all raster maps (in principle using a 
loop), using this small region and the mask, and attaching the results 
to the attribute table

6) repeat 1-5 for all polygons in the input vector

However, I noticed now that what takes time is v.rast.stats, since it 
depends on the whole region selected, which in my case is much bigger 
than a single polygon (the vector is a set of ~6,000 small polygons).


After step 4, your region should be only the size of the 
rasterized_polygon. If this is not the case there is an issue somewhere. 
If you use r.mask directly with the vector map you should be able to set 
the region to the mask using g.region raster=MASK.


Do you guys know if there is a way of using a single polygon from a 
vector to define the region using something like g.region?

[this would be really great!!]
something like
g.region vector=vector_of_interest cat=1


g.region does not have cat/where options, but I agree that this might be 
useful.


More generally, and in support of what Stefan has already said, it might 
be helpful to get a bigger picture of what exactly you are doing and 
what your data looks at exactly, as maybe looping is not your best option.


Also, keeping everything in raster and text file realm as long as 
possible could speed up the process as attribute data handling is often 
a bottle neck. You can see the code of the i.segment.stats addon [1] for 
an example of this, including parallelization of the zonal statistics of 
the different raster maps.


Moritz

[1] 
https://trac.osgeo.org/grass/browser/grass-addons/grass7/imagery/i.segment.stats/i.segment.stats.py

___
grass-user mailing list
grass-user@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] zonal statistics/metrics - beyond simple statistics

[Stefan Blumentrath]

Hi Bernado,

Please find attached the «work-in-progress» version of v.rast.bufferstats for 
some more inspiration.

v.db.select -r does what you are looking for:
https://grass.osgeo.org/grass74/manuals/v.db.select.html

However, if the polygons you are working with don`t overlap, looping should not 
be required, and you should be able to use r.univar with zones or r.stats for 
all polygons at once…

Cheers
Stefan



From: Bernardo Santos <bernardo_brand...@yahoo.com.br>
Sent: onsdag 11. april 2018 20.59
To: Helmut Kudrnovsky <hel...@web.de>; grass-user@lists.osgeo.org; Stefan 
Blumentrath <stefan.blumentr...@nina.no>
Subject: Re: [GRASS-user] zonal statistics/metrics - beyond simple statistics

Hi Helmut and Stefan,

thanks for sharing that with me, it is good to know that there are similar 
ongoing approaches to solve related issues.

Helmut,
very nice addon. However, it seems to me that the part on calculating 
statistics is still based on v.rast.stats, am I right?

Stefan, the question you pose is indeed very related. I took a better look at 
my code and I noticed that what really takes time is not the process of making 
a mask, but what comes before: the rasterization of one of the polygons of the 
input vector.
What I do is:
1) g.region using the vector layer, aligning it to the first of the input 
rasters
2) v.to.rast selecting only one of the polygons at a time from the vector layer
3) r.mask using the rasterized polygon
4) g.region zoom=rasterized_polygon
5) calculate any function for all raster maps (in principle using a loop), 
using this small region and the mask, and attaching the results to the 
attribute table
6) repeat 1-5 for all polygons in the input vector

However, I noticed now that what takes time is v.rast.stats, since it depends 
on the whole region selected, which in my case is much bigger than a single 
polygon (the vector is a set of ~6,000 small polygons).
Do you guys know if there is a way of using a single polygon from a vector to 
define the region using something like g.region?
[this would be really great!!]
something like
g.region vector=vector_of_interest cat=1

Best,
Bernardo
Em terça-feira, 10 de abril de 2018 06:01:34 BRT, Stefan Blumentrath 
<stefan.blumentr...@nina.no<mailto:stefan.blumentr...@nina.no>> escreveu:


Hei Bernardo,

Just for the record: I am currently working on something similar / related.

First of all I would like to improve speed of v.rast.stats for multiple inputs:
https://www.mail-archive.com/grass-dev@lists.osgeo.org/msg52562.html 
<https://www.mail-archive.com/grass-dev@lists.osgeo.org/msg52562.html%20> and
https://trac.osgeo.org/grass/ticket/3523

Adding a functionality to "tabulate areas" or "count categories" is a next step 
I have in mind which I am working on in v.rast.bufferstats (will be a port of 
v.what.rast.buffer [1] to GRASS 7 / pygrass (with some enhancements)). Once I 
have a good solution there I would like to see if it can be moved to 
v.rast.stats. Support for dbf driver is however a challenge...

I will add my "work in progress" script to github, so you can have a look if 
you like.

Cheers
Stefan

1: https://grass.osgeo.org/grass64/manuals/addons/

-Original Message-
From: grass-user 
<grass-user-boun...@lists.osgeo.org<mailto:grass-user-boun...@lists.osgeo.org>> 
On Behalf Of Helmut Kudrnovsky
Sent: mandag 9. april 2018 23.19
To: grass-user@lists.osgeo.org<mailto:grass-user@lists.osgeo.org>
Subject: Re: [GRASS-user] zonal statistics/metrics - beyond simple statistics

Bernardo Santos wrote
> Dear GRASS list,
> I am developing a Python script to be able to calculate (virtually
> any) metrics or statistics for zones/polygons in a vector - in analogy
> to zonal statistics (such as v.rast.stats).The idea is that one can
> calculate raster-based metrics (such as proportion of habitat, number
> of patches, or any metric that can be formalized as a function that
> takes some information from the input raster and returns a value) for
> each polygon in the vector, and this is updated as a value in a newly
> created column in the attribute table of this vector.
> Is there already anything like that (some addon/module) that I am
> missing, just to avoid re-doing something already created?
> If not, what I am doing is to create a loop over all the features in a
> vector, and for each one I zoom and use the polygon to define a mask
> (using r.mask), so that the calculation of the selected metric is
> performed only over that polygon, and this process is repeated.The
> script allows one to calculate metrics/statistics for multiple raster
> maps at once, and to incorporate other function for statistics also.
> It may be found here:https://github.com/LEEClab/GeneralizedZonalStats
>
> For small vectors this works nicely and I believe it has a great
> potential. However, when I try to calcul

Re: [GRASS-user] zonal statistics/metrics - beyond simple statistics

[Bernardo Santos]

 Hi Helmut and Stefan,
thanks for sharing that with me, it is good to know that there are similar 
ongoing approaches to solve related issues.
Helmut,very nice addon. However, it seems to me that the part on calculating 
statistics is still based on v.rast.stats, am I right?
Stefan, the question you pose is indeed very related. I took a better look at 
my code and I noticed that what really takes time is not the process of making 
a mask, but what comes before: the rasterization of one of the polygons of the 
input vector.What I do is:1) g.region using the vector layer, aligning it to 
the first of the input rasters2) v.to.rast selecting only one of the polygons 
at a time from the vector layer3) r.mask using the rasterized polygon4) 
g.region zoom=rasterized_polygon5) calculate any function for all raster maps 
(in principle using a loop), using this small region and the mask, and 
attaching the results to the attribute table6) repeat 1-5 for all polygons in 
the input vector
However, I noticed now that what takes time is v.rast.stats, since it depends 
on the whole region selected, which in my case is much bigger than a single 
polygon (the vector is a set of ~6,000 small polygons).Do you guys know if 
there is a way of using a single polygon from a vector to define the region 
using something like g.region?[this would be really great!!]something 
likeg.region vector=vector_of_interest cat=1
Best,BernardoEm terça-feira, 10 de abril de 2018 06:01:34 BRT, Stefan 
Blumentrath <stefan.blumentr...@nina.no> escreveu:  
 
 Hei Bernardo,

Just for the record: I am currently working on something similar / related.

First of all I would like to improve speed of v.rast.stats for multiple inputs:
https://www.mail-archive.com/grass-dev@lists.osgeo.org/msg52562.html and 
https://trac.osgeo.org/grass/ticket/3523

Adding a functionality to "tabulate areas" or "count categories" is a next step 
I have in mind which I am working on in v.rast.bufferstats (will be a port of 
v.what.rast.buffer [1] to GRASS 7 / pygrass (with some enhancements)). Once I 
have a good solution there I would like to see if it can be moved to 
v.rast.stats. Support for dbf driver is however a challenge...

I will add my "work in progress" script to github, so you can have a look if 
you like.

Cheers
Stefan

1: https://grass.osgeo.org/grass64/manuals/addons/

-Original Message-
From: grass-user <grass-user-boun...@lists.osgeo.org> On Behalf Of Helmut 
Kudrnovsky
Sent: mandag 9. april 2018 23.19
To: grass-user@lists.osgeo.org
Subject: Re: [GRASS-user] zonal statistics/metrics - beyond simple statistics

Bernardo Santos wrote
> Dear GRASS list,
> I am developing a Python script to be able to calculate (virtually 
> any) metrics or statistics for zones/polygons in a vector - in analogy 
> to zonal statistics (such as v.rast.stats).The idea is that one can 
> calculate raster-based metrics (such as proportion of habitat, number 
> of patches, or any metric that can be formalized as a function that 
> takes some information from the input raster and returns a value) for 
> each polygon in the vector, and this is updated as a value in a newly 
> created column in the attribute table of this vector.
> Is there already anything like that (some addon/module) that I am 
> missing, just to avoid re-doing something already created?
> If not, what I am doing is to create a loop over all the features in a 
> vector, and for each one I zoom and use the polygon to define a mask 
> (using r.mask), so that the calculation of the selected metric is 
> performed only over that polygon, and this process is repeated.The 
> script allows one to calculate metrics/statistics for multiple raster 
> maps at once, and to incorporate other function for statistics also. 
> It may be found here:https://github.com/LEEClab/GeneralizedZonalStats
> 
> For small vectors this works nicely and I believe it has a great 
> potential. However, when I try to calculate metrics for a large 
> dataset (e.g. the Brazilian map of cities, with almost 6,000 polygons) 
> - and that is when the tool would be interesting -, the process of 
> creating each mask takes too long (387 steps), and the tool becomes kind of 
> useless.
> Then I have two questions:- First, what drives the number of steps 
> GRASS takes to create a mask? Why it is very small for some maps but 
> very large for others? I quite don't understand that yet.- Do you 
> think of a easier or faster way of doing the same thing (instead of using 
> masks)?
> v.rast.stats seems to use r.univar and the option 'zones' for doing 
> so, but then one gets restricted to the statistics calculated by this module.
> Any help or comment would be very welcome!
> Best,Bernardo Niebuhr
> ___
> grass-user mailing list

> grass-user@.osgeo

Re: [GRASS-user] zonal statistics/metrics - beyond simple statistics

[Stefan Blumentrath]

Hei Bernardo,

Just for the record: I am currently working on something similar / related.

First of all I would like to improve speed of v.rast.stats for multiple inputs:
https://www.mail-archive.com/grass-dev@lists.osgeo.org/msg52562.html and 
https://trac.osgeo.org/grass/ticket/3523

Adding a functionality to "tabulate areas" or "count categories" is a next step 
I have in mind which I am working on in v.rast.bufferstats (will be a port of 
v.what.rast.buffer [1] to GRASS 7 / pygrass (with some enhancements)). Once I 
have a good solution there I would like to see if it can be moved to 
v.rast.stats. Support for dbf driver is however a challenge...

I will add my "work in progress" script to github, so you can have a look if 
you like.

Cheers
Stefan

1: https://grass.osgeo.org/grass64/manuals/addons/

-Original Message-
From: grass-user <grass-user-boun...@lists.osgeo.org> On Behalf Of Helmut 
Kudrnovsky
Sent: mandag 9. april 2018 23.19
To: grass-user@lists.osgeo.org
Subject: Re: [GRASS-user] zonal statistics/metrics - beyond simple statistics

Bernardo Santos wrote
> Dear GRASS list,
> I am developing a Python script to be able to calculate (virtually 
> any) metrics or statistics for zones/polygons in a vector - in analogy 
> to zonal statistics (such as v.rast.stats).The idea is that one can 
> calculate raster-based metrics (such as proportion of habitat, number 
> of patches, or any metric that can be formalized as a function that 
> takes some information from the input raster and returns a value) for 
> each polygon in the vector, and this is updated as a value in a newly 
> created column in the attribute table of this vector.
> Is there already anything like that (some addon/module) that I am 
> missing, just to avoid re-doing something already created?
> If not, what I am doing is to create a loop over all the features in a 
> vector, and for each one I zoom and use the polygon to define a mask 
> (using r.mask), so that the calculation of the selected metric is 
> performed only over that polygon, and this process is repeated.The 
> script allows one to calculate metrics/statistics for multiple raster 
> maps at once, and to incorporate other function for statistics also. 
> It may be found here:https://github.com/LEEClab/GeneralizedZonalStats
> 
> For small vectors this works nicely and I believe it has a great 
> potential. However, when I try to calculate metrics for a large 
> dataset (e.g. the Brazilian map of cities, with almost 6,000 polygons) 
> - and that is when the tool would be interesting -, the process of 
> creating each mask takes too long (387 steps), and the tool becomes kind of 
> useless.
> Then I have two questions:- First, what drives the number of steps 
> GRASS takes to create a mask? Why it is very small for some maps but 
> very large for others? I quite don't understand that yet.- Do you 
> think of a easier or faster way of doing the same thing (instead of using 
> masks)?
> v.rast.stats seems to use r.univar and the option 'zones' for doing 
> so, but then one gets restricted to the statistics calculated by this module.
> Any help or comment would be very welcome!
> Best,Bernardo Niebuhr
> ___
> grass-user mailing list

> grass-user@.osgeo

> https://lists.osgeo.org/mailman/listinfo/grass-user

have look at

https://grass.osgeo.org/grass74/manuals/addons/v.habitat.dem.html

where I've coded some DEM derived stats and characteristics for polygones.




-
best regards
Helmut
--
Sent from: http://osgeo-org.1560.x6.nabble.com/Grass-Users-f3884509.html
___
grass-user mailing list
grass-user@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-user
___
grass-user mailing list
grass-user@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] zonal statistics/metrics - beyond simple statistics

[Helmut Kudrnovsky]

Bernardo Santos wrote
> Dear GRASS list,
> I am developing a Python script to be able to calculate (virtually any)
> metrics or statistics for zones/polygons in a vector - in analogy to zonal
> statistics (such as v.rast.stats).The idea is that one can calculate
> raster-based metrics (such as proportion of habitat, number of patches, or
> any metric that can be formalized as a function that takes some
> information from the input raster and returns a value) for each polygon in
> the vector, and this is updated as a value in a newly created column in
> the attribute table of this vector.
> Is there already anything like that (some addon/module) that I am missing,
> just to avoid re-doing something already created?
> If not, what I am doing is to create a loop over all the features in a
> vector, and for each one I zoom and use the polygon to define a mask
> (using r.mask), so that the calculation of the selected metric is
> performed only over that polygon, and this process is repeated.The script
> allows one to calculate metrics/statistics for multiple raster maps at
> once, and to incorporate other function for statistics also. It may be
> found here:https://github.com/LEEClab/GeneralizedZonalStats
> 
> For small vectors this works nicely and I believe it has a great
> potential. However, when I try to calculate metrics for a large dataset
> (e.g. the Brazilian map of cities, with almost 6,000 polygons) - and that
> is when the tool would be interesting -, the process of creating each mask
> takes too long (387 steps), and the tool becomes kind of useless.
> Then I have two questions:- First, what drives the number of steps GRASS
> takes to create a mask? Why it is very small for some maps but very large
> for others? I quite don't understand that yet.- Do you think of a easier
> or faster way of doing the same thing (instead of using masks)?
> v.rast.stats seems to use r.univar and the option 'zones' for doing so,
> but then one gets restricted to the statistics calculated by this module.
> Any help or comment would be very welcome!
> Best,Bernardo Niebuhr
> ___
> grass-user mailing list

> grass-user@.osgeo

> https://lists.osgeo.org/mailman/listinfo/grass-user

have look at

https://grass.osgeo.org/grass74/manuals/addons/v.habitat.dem.html

where I've coded some DEM derived stats and characteristics for polygones.




-
best regards
Helmut
--
Sent from: http://osgeo-org.1560.x6.nabble.com/Grass-Users-f3884509.html
___
grass-user mailing list
grass-user@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-user

[GRASS-user] zonal statistics/metrics - beyond simple statistics

[Bernardo Santos]

Dear GRASS list,
I am developing a Python script to be able to calculate (virtually any) metrics 
or statistics for zones/polygons in a vector - in analogy to zonal statistics 
(such as v.rast.stats).The idea is that one can calculate raster-based metrics 
(such as proportion of habitat, number of patches, or any metric that can be 
formalized as a function that takes some information from the input raster and 
returns a value) for each polygon in the vector, and this is updated as a value 
in a newly created column in the attribute table of this vector.
Is there already anything like that (some addon/module) that I am missing, just 
to avoid re-doing something already created?
If not, what I am doing is to create a loop over all the features in a vector, 
and for each one I zoom and use the polygon to define a mask (using r.mask), so 
that the calculation of the selected metric is performed only over that 
polygon, and this process is repeated.The script allows one to calculate 
metrics/statistics for multiple raster maps at once, and to incorporate other 
function for statistics also. It may be found 
here:https://github.com/LEEClab/GeneralizedZonalStats

For small vectors this works nicely and I believe it has a great potential. 
However, when I try to calculate metrics for a large dataset (e.g. the 
Brazilian map of cities, with almost 6,000 polygons) - and that is when the 
tool would be interesting -, the process of creating each mask takes too long 
(387 steps), and the tool becomes kind of useless.
Then I have two questions:- First, what drives the number of steps GRASS takes 
to create a mask? Why it is very small for some maps but very large for others? 
I quite don't understand that yet.- Do you think of a easier or faster way of 
doing the same thing (instead of using masks)? v.rast.stats seems to use 
r.univar and the option 'zones' for doing so, but then one gets restricted to 
the statistics calculated by this module.
Any help or comment would be very welcome!
Best,Bernardo Niebuhr___
grass-user mailing list
grass-user@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-user