Re: [GRASS-user] problem with g.in.ogr

[Markus Metz]

On Thu, Apr 12, 2018 at 6:20 PM, Moritz Lennert <
mlenn...@club.worldonline.be> wrote:
>
> On 12/04/18 17:03, Camille Bezzina wrote:
>>
>> Hi Markus,
>>
>> In the file attached you can find the full output of v.import with
verbose.
>>
>> I launch this one with v.in.ogr and this is the answer.
>
>
> At the end you can see:
>
> "Some input polygons are overlapping each other.
> If overlapping is not desired, the data need to be cleaned.
> The input could be cleaned by snapping vertices to each other.
> Estimated range of snapping threshold: [1e-08, 1]
> Try to import again, snapping with at least 1e-08: 'snap=1e-08'"
>
> Try importing with snap=0.01 or something like that (depending on the
nature and precision of your data.

I would rather suggest snapping with a smaller value: snap=1e-4.

The main problem is close to the end, when building topology for the final
output bati_D90_decoupe:
WARNING: Nombre de contours incorrects: 8772

This is the reason for disappearing polygons. This is a bug that has been
fixed a while ago, please update your GRASS version to the latest 7.2 or
7.4 release.

Markus M

>
> Moritz
>
>>
>> Thanks.
>>
>>
>>
>> Le 12/04/2018 à 16:39, Markus Metz a écrit :
>>>
>>>
>>>
>>> On Thu, Apr 12, 2018 at 3:04 PM, Camille Bezzina <
camille.bezz...@geophom.fr > wrote:
>>> >
>>> > Hello all,
>>> >
>>> > I have a problem with v.in.ogr.
>>> > I would like to import a shape file (with 57851 polygons) with
v.in.ogr.
>>> >
>>> > v.import --verbose
input=/media/hdd1/phom/eolien/AIP_UNESCO_RONCHAMP/data/qgis/MNE/bati_D90_decoupe.shp
layer=bati_D90_decoupe output=bati_D90_decoupe -o
>>> >
>>> > Visibly the order is going well, my attribute table contains 57851
attributes but when I check my vector with display, some polygons are
missing.
>>>
>>> Can you post the full output of v.import with --verbose?
>>>
>>> Considering that you override the projection check with -o, you can
also use v.in.ogr directly and then post the full output of v.in.ogr -o
--verbose.
>>>
>>> Markus M
>>>
>>> >
>>> > Thanks for your help.
>>> >
>>> > --
>>> > Camille Bezzina
>>> > Geophom
>>> > ___
>>> > grass-user mailing list
>>> > grass-user@lists.osgeo.org 
>>> > https://lists.osgeo.org/mailman/listinfo/grass-user
>>>
>>
>> --
>> Camille BEZZINA
>> Chargé d'études géomatiques et photomontages
>> Geophom
>> 57 rue du Chemin Neuf 44521 OUDON
>> Standard: +33(0)2 85 52 02 59
>> Ligne directe: +33(0)9 72 56 81 71
>> www.geophom.fr
>>
>>
>> ___
>> grass-user mailing list
>> grass-user@lists.osgeo.org
>> https://lists.osgeo.org/mailman/listinfo/grass-user
>>
>
>
___
grass-user mailing list
grass-user@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] problem with g.in.ogr

[Moritz Lennert]

On 12/04/18 17:03, Camille Bezzina wrote:

Hi Markus,

In the file attached you can find the full output of v.import with verbose.

I launch this one with v.in.ogr and this is the answer.


At the end you can see:

"Some input polygons are overlapping each other.
If overlapping is not desired, the data need to be cleaned.
The input could be cleaned by snapping vertices to each other.
Estimated range of snapping threshold: [1e-08, 1]
Try to import again, snapping with at least 1e-08: 'snap=1e-08'"

Try importing with snap=0.01 or something like that (depending on the 
nature and precision of your data.


Moritz



Thanks.



Le 12/04/2018 à 16:39, Markus Metz a écrit :



On Thu, Apr 12, 2018 at 3:04 PM, Camille Bezzina 
> wrote:

>
> Hello all,
>
> I have a problem with v.in.ogr.
> I would like to import a shape file (with 57851 polygons) with v.in.ogr.
>
> v.import --verbose 
input=/media/hdd1/phom/eolien/AIP_UNESCO_RONCHAMP/data/qgis/MNE/bati_D90_decoupe.shp 
layer=bati_D90_decoupe output=bati_D90_decoupe -o

>
> Visibly the order is going well, my attribute table contains 57851 
attributes but when I check my vector with display, some polygons are 
missing.


Can you post the full output of v.import with --verbose?

Considering that you override the projection check with -o, you can 
also use v.in.ogr directly and then post the full output of v.in.ogr 
-o --verbose.


Markus M

>
> Thanks for your help.
>
> --
> Camille Bezzina
> Geophom
> ___
> grass-user mailing list
> grass-user@lists.osgeo.org 
> https://lists.osgeo.org/mailman/listinfo/grass-user



--
Camille BEZZINA
Chargé d'études géomatiques et photomontages
Geophom
57 rue du Chemin Neuf 44521 OUDON
Standard: +33(0)2 85 52 02 59
Ligne directe: +33(0)9 72 56 81 71
www.geophom.fr


___
grass-user mailing list
grass-user@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-user




___
grass-user mailing list
grass-user@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] zonal statistics/metrics - beyond simple statistics

[Moritz Lennert]

On 12/04/18 16:56, Bernardo Santos wrote:
For example, I am still not sure how I would use a combination of 
r.univar/r.stats and some kind of input raster map to produce, say, 
zonal info of number of habitat patches, structural or functional 
connectivity, isolation, or other more complex metrics used in landscape 
ecology.
That's way I was building something like that, so that we can just write 
an alternative function using any raster input map that returns the 
value of one of such metrics, and then use it to calculate them over 
sets of zones.


Have you looked into the landscape metrics modules, notably the r.li 
suite of modules, but also the r.pi suite in the addons (although ISTR 
that the latter is not really usable in its current state) ?


Moritz
___
grass-user mailing list
grass-user@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] zonal statistics/metrics - beyond simple statistics

[Stefan Blumentrath]

Hi Bernado,

It depends what you consider general. Even if I work with landscape ecology 
myself, I would assume landscape metrics or functional connectivity to be 
rather “special” than “general” statistics.

Putting a lot of functionality into one module can be very useful for some 
cases, but can come with a number of drawbacks as well.

A general Unix concept is to do one thing and do it well, and rather write 
modular programs that work together than very complex single tools…

Cheers
Stefan

1: https://en.wikipedia.org/wiki/Unix_philosophy

From: Bernardo Santos 
Sent: torsdag 12. april 2018 16.57
To: Helmut Kudrnovsky ; grass-user@lists.osgeo.org; Stefan 
Blumentrath 
Subject: Re: RE: [GRASS-user] zonal statistics/metrics - beyond simple 
statistics

Hi Stephan,

Some minutes before you sent me this e-mails, I'd just found out about 
v.db.select -r. It worked and made the process much faster. Thanks for that 
anyway.

In my case, the polygons do not overlap. Indeed, many things could be done with 
r.univar, r.stats or a combination of them and their output info. However, my 
aim was to think about a way of generalizing it, so that we would be able to 
use many other functions from the raster maps using the same architecture for 
calculating the zonal statistics/metrics.

For example, I am still not sure how I would use a combination of 
r.univar/r.stats and some kind of input raster map to produce, say, zonal info 
of number of habitat patches, structural or functional connectivity, isolation, 
or other more complex metrics used in landscape ecology.
That's way I was building something like that, so that we can just write an 
alternative function using any raster input map that returns the value of one 
of such metrics, and then use it to calculate them over sets of zones.

Best,
B

PS: thanks for sharing your code! This v.rast.bufferstats seems very useful, 
and I can think of some very interesting applications of that. I helped a 
friend to write something similar but using ArcPy as a ArcMap toolbox, and we 
applied it to some interesting cases in the delimitation of Protected Areas' 
buffer zones (but still did not published it). But doing that with a free and 
open source tool would be much more interesting!
Em quarta-feira, 11 de abril de 2018 18:32:18 BRT, Stefan Blumentrath 
> escreveu:



Hi Bernado,



Please find attached the «work-in-progress» version of v.rast.bufferstats for 
some more inspiration.



v.db.select -r does what you are looking for:

https://grass.osgeo.org/grass74/manuals/v.db.select.html



However, if the polygons you are working with don`t overlap, looping should not 
be required, and you should be able to use r.univar with zones or r.stats for 
all polygons at once…



Cheers

Stefan







From: Bernardo Santos 
>
Sent: onsdag 11. april 2018 20.59
To: Helmut Kudrnovsky >; 
grass-user@lists.osgeo.org; Stefan 
Blumentrath >
Subject: Re: [GRASS-user] zonal statistics/metrics - beyond simple statistics



Hi Helmut and Stefan,



thanks for sharing that with me, it is good to know that there are similar 
ongoing approaches to solve related issues.



Helmut,

very nice addon. However, it seems to me that the part on calculating 
statistics is still based on v.rast.stats, am I right?



Stefan, the question you pose is indeed very related. I took a better look at 
my code and I noticed that what really takes time is not the process of making 
a mask, but what comes before: the rasterization of one of the polygons of the 
input vector.

What I do is:

1) g.region using the vector layer, aligning it to the first of the input 
rasters

2) v.to.rast selecting only one of the polygons at a time from the vector layer

3) r.mask using the rasterized polygon

4) g.region zoom=rasterized_polygon

5) calculate any function for all raster maps (in principle using a loop), 
using this small region and the mask, and attaching the results to the 
attribute table

6) repeat 1-5 for all polygons in the input vector



However, I noticed now that what takes time is v.rast.stats, since it depends 
on the whole region selected, which in my case is much bigger than a single 
polygon (the vector is a set of ~6,000 small polygons).

Do you guys know if there is a way of using a single polygon from a vector to 
define the region using something like g.region?

[this would be really great!!]

something like

g.region vector=vector_of_interest cat=1



Best,

Bernardo

Em terça-feira, 10 de abril de 2018 06:01:34 BRT, Stefan Blumentrath 
> escreveu:





Hei Bernardo,

Just for the record: I am 

Re: [GRASS-user] problem with g.in.ogr

[Camille Bezzina]

Hi Markus,

In the file attached you can find the full output of v.import with verbose.

I launch this one with v.in.ogr and this is the answer.

Thanks.



Le 12/04/2018 à 16:39, Markus Metz a écrit :



On Thu, Apr 12, 2018 at 3:04 PM, Camille Bezzina 
> wrote:

>
> Hello all,
>
> I have a problem with v.in.ogr.
> I would like to import a shape file (with 57851 polygons) with v.in.ogr.
>
> v.import --verbose 
input=/media/hdd1/phom/eolien/AIP_UNESCO_RONCHAMP/data/qgis/MNE/bati_D90_decoupe.shp 
layer=bati_D90_decoupe output=bati_D90_decoupe -o

>
> Visibly the order is going well, my attribute table contains 57851 
attributes but when I check my vector with display, some polygons are 
missing.


Can you post the full output of v.import with --verbose?

Considering that you override the projection check with -o, you can 
also use v.in.ogr directly and then post the full output of v.in.ogr 
-o --verbose.


Markus M

>
> Thanks for your help.
>
> --
> Camille Bezzina
> Geophom
> ___
> grass-user mailing list
> grass-user@lists.osgeo.org 
> https://lists.osgeo.org/mailman/listinfo/grass-user



--
Camille BEZZINA
Chargé d'études géomatiques et photomontages
Geophom
57 rue du Chemin Neuf 44521 OUDON
Standard: +33(0)2 85 52 02 59
Ligne directe: +33(0)9 72 56 81 71
www.geophom.fr
(Thu Apr 12 12:45:14 2018)  
v.import --verbose 
input=/media/hdd1/phom/eolien/AIP_UNESCO_RONCHAMP/data/qgis/MNE/bati_D90_decoupe.shp
 layer=bati_D90_decoupe output=bati_D90_decoupe -o
Creating temporary location for 
...
WARNING: Datum  non reconnu par GRASS et 
aucuns paramètres trouvés
WARNING: Datum  non reconnu par GRASS et 
aucuns paramètres trouvés
Over-riding projection check
Check if OGR layer  contains polygons...
Boundary splitting distance in map units: 201.589
Utilisation du format natif
Utilisation du format natif
Using temporary vector 
Importing 57851 features (OGR layer )...
-
Enregistrement des primitives ...
58614 primitives registered
471329 vertices registered
La topologie a bien été construite
Nombre de nœuds: 55752
Nombre de primitives: 58614
Nombre de points: 0
Nombre de lignes :0
Nombre de contours: 58614
Nombre de centroïdes: 0
Nombre de surfaces: -
Nombre d'îles: -
-
Cleaning polygons
-
Snapping boundaries (threshold = 1.000e-13)...
Reading features...
Accrochage des sommets passe 1 : sélection des points
Accrochage des sommets passe 2 : assignation des ancrages de sommets
Accrochage des sommets passe 3 : accrochage sur les points assignés
Sommets accrochés : 0
Nouveau sommet : 0
-
Breaking polygons...
Breaking polygons (pass 1: select break points)...
Breaking polygons (pass 2: break at selected points)...
Breaks: 45047
-
Removing duplicates...
Removed duplicates: 2208
-
Breaking boundaries...
Intersections : 10
-
Removing duplicates...
Removed duplicates: 4
-
Cleaning boundaries at nodes...
Modifications : 31992
-
Breaking boundaries...
Intersections : 18
-
Removing duplicates...
Removed duplicates: 1064
-
Cleaning boundaries at nodes...
Modifications : 1104
-
Breaking boundaries...
Intersections : 17
-
Removing duplicates...
Removed duplicates: 17
-
Cleaning boundaries at nodes...
Modifications : 0
-
Merging boundaries...
11418 contours fusionnés
5601 nouveaux contours
-
Removing dangles...
Supprimé lignes : 0
Supprimé dangles: 0
-
Construction des surfaces ...
58043 areas built
47941 isles built
La topologie a bien été construite
Nombre de nœuds: 69634
Nombre de primitives: 121114
Nombre de points: 0
Nombre de lignes :0
Nombre de contours: 121114
Nombre de centroïdes: 0
Nombre de surfaces: 58043
Nombre d'îles: 47941
WARNING: Nombre de contours incorrects: 8776
-
Removing bridges...
Removed lines: 3
Removed bridges: 1
-
Enregistrement des primitives ...
82317 primitives registered
490544 vertices registered
Construction des surfaces ...
46766 

Re: [GRASS-user] zonal statistics/metrics - beyond simple statistics

[Bernardo Santos]

 Hi Stephan,

Some minutes before you sent me this e-mails, I'd just found out about 
v.db.select -r. It worked and made the process much faster. Thanks for that 
anyway.
In my case, the polygons do not overlap. Indeed, many things could be done with 
r.univar, r.stats or a combination of them and their output info. However, my 
aim was to think about a way of generalizing it, so that we would be able to 
use many other functions from the raster maps using the same architecture for 
calculating the zonal statistics/metrics.
For example, I am still not sure how I would use a combination of 
r.univar/r.stats and some kind of input raster map to produce, say, zonal info 
of number of habitat patches, structural or functional connectivity, isolation, 
or other more complex metrics used in landscape ecology.That's way I was 
building something like that, so that we can just write an alternative function 
using any raster input map that returns the value of one of such metrics, and 
then use it to calculate them over sets of zones.
Best,B
PS: thanks for sharing your code! This v.rast.bufferstats seems very useful, 
and I can think of some very interesting applications of that. I helped a 
friend to write something similar but using ArcPy as a ArcMap toolbox, and we 
applied it to some interesting cases in the delimitation of Protected Areas' 
buffer zones (but still did not published it). But doing that with a free and 
open source tool would be much more interesting! Em quarta-feira, 11 de 
abril de 2018 18:32:18 BRT, Stefan Blumentrath  
escreveu:  
 
 #yiv6359193583 #yiv6359193583 -- _filtered #yiv6359193583 
{font-family:Helvetica;panose-1:2 11 6 4 2 2 2 2 2 4;} _filtered #yiv6359193583 
{panose-1:2 4 5 3 5 4 6 3 2 4;} _filtered #yiv6359193583 
{font-family:Calibri;panose-1:2 15 5 2 2 2 4 3 2 4;}#yiv6359193583 
#yiv6359193583 p.yiv6359193583MsoNormal, #yiv6359193583 
li.yiv6359193583MsoNormal, #yiv6359193583 div.yiv6359193583MsoNormal 
{margin:0cm;margin-bottom:.0001pt;font-size:12.0pt;font-family:New 
serif;}#yiv6359193583 a:link, #yiv6359193583 span.yiv6359193583MsoHyperlink 
{color:blue;text-decoration:underline;}#yiv6359193583 a:visited, #yiv6359193583 
span.yiv6359193583MsoHyperlinkFollowed 
{color:purple;text-decoration:underline;}#yiv6359193583 
p.yiv6359193583msonormal0, #yiv6359193583 li.yiv6359193583msonormal0, 
#yiv6359193583 div.yiv6359193583msonormal0 
{margin-right:0cm;margin-left:0cm;font-size:12.0pt;font-family:New 
serif;}#yiv6359193583 span.yiv6359193583EmailStyle18 
{font-family:sans-serif;color:#1F497D;}#yiv6359193583 
.yiv6359193583MsoChpDefault {font-size:10.0pt;} _filtered #yiv6359193583 
{margin:70.85pt 70.85pt 70.85pt 70.85pt;}#yiv6359193583 
div.yiv6359193583WordSection1 {}#yiv6359193583 
Hi Bernado,
 
  
 
Please find attached the «work-in-progress» version of v.rast.bufferstats for 
some more inspiration.
 
  
 
v.db.select -r does what you are looking for:
 
https://grass.osgeo.org/grass74/manuals/v.db.select.html
 
  
 
However, if the polygons you are working with don`t overlap, looping should not 
be required, and you should be able to use r.univar with zones or r.stats for 
all polygons at once…
 
  
 
Cheers
 
Stefan
 
  
 
  
 
  
 
From: Bernardo Santos 
Sent: onsdag 11. april 2018 20.59
To: Helmut Kudrnovsky ; grass-user@lists.osgeo.org; Stefan 
Blumentrath 
Subject: Re: [GRASS-user] zonal statistics/metrics - beyond simple statistics
 
  
 
Hi Helmut and Stefan,
 
  
 
thanks for sharing that with me, it is good to know that there are 
similarongoing approaches to solve related issues.
 
  
 
Helmut,
 
very nice addon. However, it seems to me that the part on calculating 
statistics is still based on v.rast.stats, am I right?
 
  
 
Stefan, the question you pose is indeed very related. I took a better look at 
my code and I noticed that what really takes time is not the process of making 
a mask, but what comes before: the rasterization of one of the polygons of the 
input vector.
 
What I do is:
 
1) g.region using the vector layer, aligning it to the first of the input 
rasters
 
2) v.to.rast selecting only one of the polygons at a time from the vector layer
 
3) r.mask using the rasterized polygon
 
4) g.region zoom=rasterized_polygon
 
5) calculate any function for all raster maps (in principle using a loop), 
using this small region and the mask, and attaching the results to the 
attribute table
 
6) repeat 1-5 for all polygons in the input vector
 
  
 
However, I noticed now that what takes time is v.rast.stats, since it depends 
on the whole region selected, which in my case is much bigger than a single 
polygon (the vector is a set of ~6,000 small polygons).
 
Do you guys know if there is a way of using a single polygon from a vector to 
define the region using something like g.region?
 
[this would be really great!!]
 
something like
 
g.region 

Re: [GRASS-user] problem with g.in.ogr

[Markus Metz]

On Thu, Apr 12, 2018 at 3:04 PM, Camille Bezzina 
wrote:
>
> Hello all,
>
> I have a problem with v.in.ogr.
> I would like to import a shape file (with 57851 polygons) with v.in.ogr.
>
> v.import --verbose
input=/media/hdd1/phom/eolien/AIP_UNESCO_RONCHAMP/data/qgis/MNE/bati_D90_decoupe.shp
layer=bati_D90_decoupe output=bati_D90_decoupe -o
>
> Visibly the order is going well, my attribute table contains 57851
attributes but when I check my vector with display, some polygons are
missing.

Can you post the full output of v.import with --verbose?

Considering that you override the projection check with -o, you can also
use v.in.ogr directly and then post the full output of v.in.ogr -o
--verbose.

Markus M

>
> Thanks for your help.
>
> --
> Camille Bezzina
> Geophom
> ___
> grass-user mailing list
> grass-user@lists.osgeo.org
> https://lists.osgeo.org/mailman/listinfo/grass-user
___
grass-user mailing list
grass-user@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-user

[GRASS-user] problem with g.in.ogr

[Camille Bezzina]

Hello all,

I have a problem with v.in.ogr.
I would like to import a shape file (with 57851 polygons) with v.in.ogr.

v.import --verbose 
input=/media/hdd1/phom/eolien/AIP_UNESCO_RONCHAMP/data/qgis/MNE/bati_D90_decoupe.shp 
layer=bati_D90_decoupe output=bati_D90_decoupe -o


Visibly the order is going well, my attribute table contains 57851 
attributes but when I check my vector with display, some polygons are 
missing.


Thanks for your help.

--
Camille Bezzina
Geophom
___
grass-user mailing list
grass-user@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] zonal statistics/metrics - beyond simple statistics

[Moritz Lennert]

On 11/04/18 20:58, Bernardo Santos wrote:

Hi Helmut and Stefan,

thanks for sharing that with me, it is good to know that there are 
similar ongoing approaches to solve related issues.


Helmut,
very nice addon. However, it seems to me that the part on calculating 
statistics is still based on v.rast.stats, am I right?


Stefan, the question you pose is indeed very related. I took a better 
look at my code and I noticed that what really takes time is not the 
process of making a mask, but what comes before: the rasterization of 
one of the polygons of the input vector.

What I do is:
1) g.region using the vector layer, aligning it to the first of the 
input rasters
2) v.to.rast selecting only one of the polygons at a time from the 
vector layer

3) r.mask using the rasterized polygon


You can actually use r.mask directly with the vector map using the 
vector and cat/where parameters.



4) g.region zoom=rasterized_polygon
5) calculate any function for all raster maps (in principle using a 
loop), using this small region and the mask, and attaching the results 
to the attribute table

6) repeat 1-5 for all polygons in the input vector

However, I noticed now that what takes time is v.rast.stats, since it 
depends on the whole region selected, which in my case is much bigger 
than a single polygon (the vector is a set of ~6,000 small polygons).


After step 4, your region should be only the size of the 
rasterized_polygon. If this is not the case there is an issue somewhere. 
If you use r.mask directly with the vector map you should be able to set 
the region to the mask using g.region raster=MASK.


Do you guys know if there is a way of using a single polygon from a 
vector to define the region using something like g.region?

[this would be really great!!]
something like
g.region vector=vector_of_interest cat=1


g.region does not have cat/where options, but I agree that this might be 
useful.


More generally, and in support of what Stefan has already said, it might 
be helpful to get a bigger picture of what exactly you are doing and 
what your data looks at exactly, as maybe looping is not your best option.


Also, keeping everything in raster and text file realm as long as 
possible could speed up the process as attribute data handling is often 
a bottle neck. You can see the code of the i.segment.stats addon [1] for 
an example of this, including parallelization of the zonal statistics of 
the different raster maps.


Moritz

[1] 
https://trac.osgeo.org/grass/browser/grass-addons/grass7/imagery/i.segment.stats/i.segment.stats.py

___
grass-user mailing list
grass-user@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-user