Re: [GRASS-user] V.generalize

[DAZIN Fabrice (SIRS)]

Hi,

Yes i try with other methods of v.generalize tool with same problems.
I realize that if the generalisation tool creates a topological error on a part 
of the vector then the tool does not treat this part and leaves it as it is.
I didn't succeed in forcing him to do it.

As a workaround i used https://mapshaper.org/   that implement the Visvalingam 
algo that work fine with a real time result!
(see 
https://gis.stackexchange.com/questions/87505/grass-v-generalize-method-douglas-not-working-as-expected)

My challenge is now to create a tool in grass with this Visvalingam algo.

Best regards
Fabrice Dazin
[cid:8590fe01-f45d-4f0d-8c6a-8ce2a9867149]

De : Markus Neteler 
Envoyé : mercredi 26 août 2020 16:27
À : DAZIN Fabrice (SIRS) 
Cc : GRASS user list 
Objet : Re: [GRASS-user] V.generalize

Hi,

On Mon, Aug 24, 2020 at 4:35 PM DAZIN Fabrice (SIRS)
 wrote:
>
>
> Dear all,
>
> In the process of transforming a raster classification into a vector layer, I 
> use among others the command V.GENERALIZE.
>
> v.generalize input=v_classif_ext@PERMANENT layer=1 type=area type=boundary  
> method=reduction threshold=4 alpha=0.5 iterations=4 
> output=v_classif_ext_redv.generalize input=v_classif_ext_red@PERMANENT 
> layer=1 type=area type=boundary method=sliding_averaging threshold=100 
> alpha=0.5 iterations=4 output=v_smooth_1
>
> with
> grassversion=79
> srid=2154
>
> And I wonder why the smoothing result is not homogeneous, even sometimes for 
> the same polygon.
> Any idea about the reason?

Not sure about this particular "reduction" method but did you already
try with a different method?

Best regards,
Markus

--
Markus Neteler, PhD
https://m365.eu.vadesecure.com/safeproxy/v3?f=RvgVPrGFK9ImWkc-9ucPkT9AzWPavDwg5ArpeNZO76GHNfmu9lpEhdAB9mPfAIPd=QLTB8Xx5rEin-ejf6tcfqHNZKtEZCTz67dt9z7tSA_LOvn7_adzZnF75USR2DAN00a-e9VK7JN6-lYZ8w2hZTQ=y2yg=HEfuyjLfWxXkfJp6GHFMLpwbQoGmoxACm10xKlkrlyQFjw7woCT5r4dUIFgcq8FO=https%3A%2F%2Fwww.mundialis.de
 - free data with free software
https://m365.eu.vadesecure.com/safeproxy/v3?f=58-viv3m7Zpv529C2dqB0DyOlDDgVIsuSqAqdZObHwfQEoUb5YWRMJyxmUPZVNpW=YR0xURJMeg8eaqd82mBKKAkbIOdoE-7ec56BwNI59hNv-a0JutacywyaIc3Mim0Rj_XfDpKHgjufUUek5owIQA=gGQO=QTd8p84Y91HNOcnftrP7SROBtekUcvegEtbqD7srCXHqjXgqyC_JwCaDEv-zoucw=https%3A%2F%2Fgrass.osgeo.org
https://m365.eu.vadesecure.com/safeproxy/v3?f=UlBtn-GsC69yUrXb8YoleYNVYehGazxYM92ZFD5292MvRueEqmNEpLZ2LIMi1nHp=1t2-Qxk4t1wp9x95oA6Ik3voCjEeNJDUr3iach3VFY0Exl_nYRUiOLNK4rUz8-0c19EflCq2lX_Cp8Vv7pmnsw=EnXZ=h41QrHhrJELVMIbNOiWM2PZIcDewreyiiVUdqyJ3tPeVDJ38TJH7IxO66ceBiYHz=https%3A%2F%2Fcourses.neteler.org%2Fblog


Ce message et toutes les pièces jointes (ci-après le "message") sont établis à 
l'intention exclusive de ses destinataires et sont confidentiels. Si vous 
recevez ce message par erreur ou s'il ne vous est pas destiné, merci de le 
détruire ainsi que toute copie de votre système et d'en avertir immédiatement 
l'expéditeur. Toute lecture non autorisée, toute utilisation de ce message qui 
n'est pas conforme à sa destination, toute diffusion ou toute publication, 
totale ou partielle, est interdite. L'Internet ne permettant pas d'assurer 
l'intégrité de ce message électronique susceptible d'altération, l'expéditeur 
(et ses filiales) décline(nt) toute responsabilité au titre de ce message dans 
l'hypothèse où il aurait été modifié ou falsifié.

This message and any attachments (the "message") is intended solely for the 
intended recipient(s) and is confidential. If you receive this message in 
error, or are not the intended recipient(s), please delete it and any copies 
from your systems and immediately notify the sender. Any unauthorized view, use 
that does not comply with its purpose, dissemination or disclosure, either 
whole or partial, is prohibited. Since the internet cannot guarantee the 
integrity of this message which may not be reliable, the sender (and its 
subsidiaries) shall not be liable for the message if modified or falsified.
___
grass-user mailing list
grass-user@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] V.generalize

[Markus Neteler]

Hi,

On Mon, Aug 24, 2020 at 4:35 PM DAZIN Fabrice (SIRS)
 wrote:
>
>
> Dear all,
>
> In the process of transforming a raster classification into a vector layer, I 
> use among others the command V.GENERALIZE.
>
> v.generalize input=v_classif_ext@PERMANENT layer=1 type=area type=boundary  
> method=reduction threshold=4 alpha=0.5 iterations=4 
> output=v_classif_ext_redv.generalize input=v_classif_ext_red@PERMANENT 
> layer=1 type=area type=boundary method=sliding_averaging threshold=100 
> alpha=0.5 iterations=4 output=v_smooth_1
>
> with
> grassversion=79
> srid=2154
>
> And I wonder why the smoothing result is not homogeneous, even sometimes for 
> the same polygon.
> Any idea about the reason?

Not sure about this particular "reduction" method but did you already
try with a different method?

Best regards,
Markus

-- 
Markus Neteler, PhD
https://www.mundialis.de - free data with free software
https://grass.osgeo.org
https://courses.neteler.org/blog
___
grass-user mailing list
grass-user@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] v.generalize and where clause

[Micha Silver]

  
  

On 22/04/2019 01:35, Markus Metz wrote:


  
  

On Sun, Apr 21, 2019 at 11:16 AM Micha Silver 
wrote:
>
>
>
> On 21/04/2019 0:42, Markus Metz wrote:
>

>
> "If type=area is selected, boundaries of selected areas
will be generalized, and the options cats, where, and layer will
be used to select areas. However the cats and where options are
used only if a layer is specified. If no layer is specified,
those options are ignored."


The cats, where, and layer options apply to areas as well
  as lines.


Therefore I have added
"The cats and where options are used only
  if a layer > 0 is specified, otherwise, those
  options are ignored. Be aware that the default is layer=-1,
  meaning that all layers are processed, ignoring the cats
  and where options."


to the manual in trunk and relbr76.



Markus M

>
  



Great, thanks

-- 
Micha Silver
Ben Gurion Univ.
Sde Boker, Remote Sensing Lab
+972-523-665918
  

___
grass-user mailing list
grass-user@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] v.generalize and where clause

[Markus Metz]

On Sun, Apr 21, 2019 at 11:16 AM Micha Silver  wrote:
>
>
>
> On 21/04/2019 0:42, Markus Metz wrote:
>
> >
> > Am I missing something?
>
> Yes, the option layer defaults to -1 = all layers. If you use the cats or
where option, you need to specify a layer, otherwise the warning
> WARNING: No layer selected, 'where' and 'cats' options are ignored
>
>
> Ah ha
>
> Thanks,
>
>
> Maybe an extra sentence in the manpage would be appropriate?
>
> Under the "Description" section:
>
>
> "If type=area is selected, boundaries of selected areas will be
generalized, and the options cats, where, and layer will be used to select
areas. However the cats and where options are used only if a layer is
specified. If no layer is specified, those options are ignored."

The cats, where, and layer options apply to areas as well as lines.

Therefore I have added
"The *cats* and *where* options are used only if a *layer* > 0 is
specified, otherwise, those options are ignored. Be aware that the default
is *layer=-1*, meaning that all layers are processed, ignoring the *cats*
and *where* options."

to the manual in trunk and relbr76.

Markus M
>
> Best, Micha
>
>
> is issued in trunk. I have backported this warning to relbr76 in r74409.
>
> HTH,
>
> Markus M
>
> --
> Micha Silver
> Ben Gurion Univ.
> Sde Boker, Remote Sensing Lab
> cell: +972-523-665918
___
grass-user mailing list
grass-user@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] v.generalize and where clause

[Micha Silver]

  
  



On 21/04/2019 0:42, Markus Metz wrote:

>
  > Am I missing something?
  
  
  Yes, the option layer defaults to -1 = all layers. If you use
the cats or where option, you need to specify a layer, otherwise
the warning
  WARNING: No layer selected, 'where' and 'cats' options are
ignored



Ah ha
Thanks,


Maybe an extra sentence in the manpage would be appropriate?
Under the "Description" section:


"If type=area is selected, boundaries of selected areas
  will be generalized, and the options cats, where,
  and layer will be used to select areas. However the cats
  and where options are used only if a layer is
  specified. If no layer is specified, those options are ignored."


Best, Micha


  
  
  is issued in trunk. I have backported this warning to relbr76
in r74409.
  
  
  HTH,
  
  
  Markus M

-- 
Micha Silver
Ben Gurion Univ.
Sde Boker, Remote Sensing Lab
cell: +972-523-665918
  

___
grass-user mailing list
grass-user@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] v.generalize and where clause

[Markus Metz]

On Sat, Apr 20, 2019 at 8:27 PM Micha Silver  wrote:
>
> Trying to address this SE question:
>
>
>
https://gis.stackexchange.com/questions/319319/how-to-apply-smoothing-filter-only-to-way-bigger-than-specified-number-of-nodes
>
>
> I tried to apply v.generalize to a subset of area features, using a where
clause, but it doesn't seem to be working. All features are generlized, as
if the where clause is being ignored. Here's an example from the
nc_basic_sm MAPSET:
>
>
> micha@TP480:~$ v.extract geology output=my_geology --o
where="GEO_NAME='Qp'"
> WARNING: Vector map  already exists and will be overwritten
>
> .
>
>
> micha@TP480:~$ v.db.addcolumn my_geology column="area_sqkm DOUBLE"
>
> micha@TP480:~$ v.to.db my_geology option=area column=area_sqkm
unit=kilometer
> WARNING: Values in column  will be overwritten
> .
> 326 records updated/inserted (layer 1)
>
>
> # Only 21 features are larger than 10 sq.km.:
>
> micha@TP480:~$  v.db.select -c my_geology column=area_sqkm
where="area_sqkm>10" | wc -l
> 21
>
>
> micha@TP480:~$ v.generalize my_geology output=my_geology_smooth type=area
method=douglas thresh=100 where="area_sqkm>10" --o
>
> 
>
> v.generalize complete. Number of vertices for selected features reduced
> from 44238 to 7540 (17% remaining)
>
>
> Now viewing both my_geology and my_geology_smooth shows that *all*
features were smoothed. Including those island polygons that were much
smaller than 10 sq.km.
>
>
> Am I missing something?

Yes, the option layer defaults to -1 = all layers. If you use the cats or
where option, you need to specify a layer, otherwise the warning
WARNING: No layer selected, 'where' and 'cats' options are ignored

is issued in trunk. I have backported this warning to relbr76 in r74409.

HTH,

Markus M
___
grass-user mailing list
grass-user@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] v.generalize and external boundaries

[Daniel McInerney]

Hi Markus, Micha,

Many thanks for your emails, the combined solution that you suggested
worked perfectly.

best regards,

Daniel.


On 25/08/18 11:03, Markus Metz wrote:
>
>
> On Sat, Aug 25, 2018 at 10:56 AM Micha Silver  > wrote:
> >
> > I think I have a series of steps that will allow you to separate the
> "inner" boundaries and generalize only those. The idea is based on
> using the v.to.db module, with the "sides" option to add the cat
> number of each area to the left and right of each boundary. Outer
> boundaries will have "-1" (nothing on that side) for the left or right
> sides. Then you can split out the inner boundaries with a v.extract
> using a "where" clause.
>
> nice approach, but it could be simplified because v.generalize has a
> where option, i.e. no need for v.extract:
> >
> > Somewhat convoluted, but here you are as regular grass command, not
> in python.
> >
> > (My example below is from the sample nc_basic_spm_grass7 location. I
> tested with the geology map.)
> >
> > # Make a copy and add cats to the boundaries in layer 2
> > GRASS 7.4.0 (nc_basic_spm_grass7):~ > g.copy vect=geology,my_geol
> > GRASS 7.4.0 (nc_basic_spm_grass7):~ > v.category my_geol layer=2
> type=boundary option=add output=my_geol2
> >
> > # Add attribute columns for the left and right sides in layer 2, and
> populate from the areas in *layer 1*
> > GRASS 7.4.0 (nc_basic_spm_grass7):~ > v.db.addtable my_geol2 layer=2
> column="left integer,right integer"
> > GRASS 7.4.0 (nc_basic_spm_grass7):~ > v.to.db my_geol2 layer=2
> option=sides column=left,right query_layer=1
>
> generalize inner boundaries
> GRASS 7.4.0 (nc_basic_spm_grass7):~ > v.generalize input=my_geol2
> method=douglas output=my_geol2_smooth thresh=1000 type=boundary
> layer=2 where="left > -1 and right > -1" --o
>
> done
>
> Markus M
>
> >
> > # Now use the left and right cat value to separate the inner and
> outer boundaries
> > GRASS 7.4.0 (nc_basic_spm_grass7):~ > v.extract my_geol2 layer=2
> out=inner_bndry type=boundary where="left<>-1 and right<>-1"
> > GRASS 7.4.0 (nc_basic_spm_grass7):~ > v.extract my_geol2 layer=2
> out=outer_bndry type=boundary where="left=-1 or right=-1"
> >
> > # Convert inner boundaries to lines for smoothing, and run generalize
> > GRASS 7.4.0 (nc_basic_spm_grass7):~ > v.type input=inner_bndry
> out=inner_lines layer=2 from=boundary to=line
> > GRASS 7.4.0 (nc_basic_spm_grass7):~ > v.generalize inner_lines
> method=douglas output=inner_lines_smooth thresh=1000 --o
> >
> > # Convert back to boundary and merge back with the outer boundary
> and recreate area centroids
> > GRASS 7.4.0 (nc_basic_spm_grass7):~ > v.type inner_lines_smooth
> out=inner_bndry_smooth from=line to=boundary layer=2 --o
> > GRASS 7.4.0 (nc_basic_spm_grass7):~ > v.patch
> input=outer_bndry,inner_bndry_smooth output=merge_smooth --o
> > GRASS 7.4.0 (nc_basic_spm_grass7):~ > v.centroids merge_smooth
> output=merged_areas layer=2 option=add
> >
> >
> > You probably will have to do some manual cleaning of the result,
> since the smoothed inner boundaries might no longer exactly intersect
> with the untouched outer boundary due to overshoots, etc.
> >
> > If you try this, let us know how it went.
> >
> > Regards, Micha
> >
> >
> >
> > On 08/24/2018 06:43 PM, Daniel McInerney wrote:
> >
> > Hi List,
> >
> > I'm using v.generalize in a script (excerpt included below) to smooth
> > the boundaries of a polygon vector dataset. In the attached example
> > (input_vector.png), I would like to *only* generalize the internal lines
> > (blue), and leave the external boundary (green line) unchanged. I
> > thought that the flag, '-l' might provide this functionality, but
> > unfortunately the resulting external boundary is changed significantly
> > (attached vgeneralize_output.png).
> >
> > Is this functionality available in v.generalize or is there another way
> > that this could be achieved in GRASS?
> >
> > Thanks in advance.
> >
> > Best regards,
> > Daniel.
> >
> > ##run generalisation (step: 1 - douglas)
> > gscript.run_command('v.generalize', flags='l', overwrite=True,
> > input='segments', output='segments_douglas', method='douglas',
> threshold=1)
> >
> > ##run generalisation (step: 2 - sliding average)
> > gscript.run_command('v.generalize', flags='l', overwrite=True,
> > input='segments_douglas', output='segments_douglas_slide',
> > method='sliding_averaging', threshold='2', look_ahead='9', slide='0.1',
> > iterations='3')
> >
> > ##run generalisation (step: 3 - snake)
> > gscript.run_command('v.generalize', flags='l', overwrite=True,
> > input='segments_douglas_slide', output='segments_douglas_slide_snake',
> > method='snake', threshold='3', alpha='1', beta='1')
> >  
> >  
> >
> >
> >
> > ___
> > grass-user mailing list
> > grass-user@lists.osgeo.org 
> > https://lists.osgeo.org/mailman/listinfo/grass-user
> >
> >
> > --
> > Micha Silver
> > Ben Gurion 

Re: [GRASS-user] v.generalize and external boundaries

[Markus Metz]

On Sat, Aug 25, 2018 at 10:56 AM Micha Silver  wrote:
>
> I think I have a series of steps that will allow you to separate the
"inner" boundaries and generalize only those. The idea is based on using
the v.to.db module, with the "sides" option to add the cat number of each
area to the left and right of each boundary. Outer boundaries will have
"-1" (nothing on that side) for the left or right sides. Then you can split
out the inner boundaries with a v.extract using a "where" clause.

nice approach, but it could be simplified because v.generalize has a where
option, i.e. no need for v.extract:
>
> Somewhat convoluted, but here you are as regular grass command, not in
python.
>
> (My example below is from the sample nc_basic_spm_grass7 location. I
tested with the geology map.)
>
> # Make a copy and add cats to the boundaries in layer 2
> GRASS 7.4.0 (nc_basic_spm_grass7):~ > g.copy vect=geology,my_geol
> GRASS 7.4.0 (nc_basic_spm_grass7):~ > v.category my_geol layer=2
type=boundary option=add output=my_geol2
>
> # Add attribute columns for the left and right sides in layer 2, and
populate from the areas in *layer 1*
> GRASS 7.4.0 (nc_basic_spm_grass7):~ > v.db.addtable my_geol2 layer=2
column="left integer,right integer"
> GRASS 7.4.0 (nc_basic_spm_grass7):~ > v.to.db my_geol2 layer=2
option=sides column=left,right query_layer=1

generalize inner boundaries
GRASS 7.4.0 (nc_basic_spm_grass7):~ > v.generalize input=my_geol2
method=douglas output=my_geol2_smooth thresh=1000 type=boundary layer=2
where="left > -1 and right > -1" --o

done

Markus M

>
> # Now use the left and right cat value to separate the inner and outer
boundaries
> GRASS 7.4.0 (nc_basic_spm_grass7):~ > v.extract my_geol2 layer=2
out=inner_bndry type=boundary where="left<>-1 and right<>-1"
> GRASS 7.4.0 (nc_basic_spm_grass7):~ > v.extract my_geol2 layer=2
out=outer_bndry type=boundary where="left=-1 or right=-1"
>
> # Convert inner boundaries to lines for smoothing, and run generalize
> GRASS 7.4.0 (nc_basic_spm_grass7):~ > v.type input=inner_bndry
out=inner_lines layer=2 from=boundary to=line
> GRASS 7.4.0 (nc_basic_spm_grass7):~ > v.generalize inner_lines
method=douglas output=inner_lines_smooth thresh=1000 --o
>
> # Convert back to boundary and merge back with the outer boundary and
recreate area centroids
> GRASS 7.4.0 (nc_basic_spm_grass7):~ > v.type inner_lines_smooth
out=inner_bndry_smooth from=line to=boundary layer=2 --o
> GRASS 7.4.0 (nc_basic_spm_grass7):~ > v.patch
input=outer_bndry,inner_bndry_smooth output=merge_smooth --o
> GRASS 7.4.0 (nc_basic_spm_grass7):~ > v.centroids merge_smooth
output=merged_areas layer=2 option=add
>
>
> You probably will have to do some manual cleaning of the result, since
the smoothed inner boundaries might no longer exactly intersect with the
untouched outer boundary due to overshoots, etc.
>
> If you try this, let us know how it went.
>
> Regards, Micha
>
>
>
> On 08/24/2018 06:43 PM, Daniel McInerney wrote:
>
> Hi List,
>
> I'm using v.generalize in a script (excerpt included below) to smooth
> the boundaries of a polygon vector dataset. In the attached example
> (input_vector.png), I would like to *only* generalize the internal lines
> (blue), and leave the external boundary (green line) unchanged. I
> thought that the flag, '-l' might provide this functionality, but
> unfortunately the resulting external boundary is changed significantly
> (attached vgeneralize_output.png).
>
> Is this functionality available in v.generalize or is there another way
> that this could be achieved in GRASS?
>
> Thanks in advance.
>
> Best regards,
> Daniel.
>
> ##run generalisation (step: 1 - douglas)
> gscript.run_command('v.generalize', flags='l', overwrite=True,
> input='segments', output='segments_douglas', method='douglas',
threshold=1)
>
> ##run generalisation (step: 2 - sliding average)
> gscript.run_command('v.generalize', flags='l', overwrite=True,
> input='segments_douglas', output='segments_douglas_slide',
> method='sliding_averaging', threshold='2', look_ahead='9', slide='0.1',
> iterations='3')
>
> ##run generalisation (step: 3 - snake)
> gscript.run_command('v.generalize', flags='l', overwrite=True,
> input='segments_douglas_slide', output='segments_douglas_slide_snake',
> method='snake', threshold='3', alpha='1', beta='1')
>
>
>
>
>
> ___
> grass-user mailing list
> grass-user@lists.osgeo.org
> https://lists.osgeo.org/mailman/listinfo/grass-user
>
>
> --
> Micha Silver
> Ben Gurion Univ.
> Sde Boker, Remote Sensing Lab
> cell: +972-523-665918
>
> ___
> grass-user mailing list
> grass-user@lists.osgeo.org
> https://lists.osgeo.org/mailman/listinfo/grass-user
___
grass-user mailing list
grass-user@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] v.generalize and external boundaries

[Micha Silver]

  
  
I think I have a series of steps that will allow you to separate the
"inner" boundaries and generalize only those. The idea is based on
using the v.to.db module, with the "sides" option to add the cat
number of each area to the left and right of each boundary. Outer
boundaries will have "-1" (nothing on that side) for the left or
right sides. Then you can split out the inner boundaries with a
v.extract using a "where" clause.

Somewhat convoluted, but here you are as regular grass command, not
in python.

(My example below is from the sample nc_basic_spm_grass7 location. I
tested with the geology map.)

# Make a copy and add cats to the boundaries in
layer 2
  GRASS 7.4.0 (nc_basic_spm_grass7):~ > g.copy
vect=geology,my_geol
  GRASS 7.4.0 (nc_basic_spm_grass7):~ > v.category
my_geol layer=2 type=boundary option=add output=my_geol2
  
# Add attribute columns for the left and right sides in layer 2,
and populate from the areas in *layer 1*
  GRASS 7.4.0 (nc_basic_spm_grass7):~ > v.db.addtable
my_geol2 layer=2 column="left integer,right integer"
  GRASS 7.4.0 (nc_basic_spm_grass7):~ > v.to.db my_geol2
layer=2 option=sides column=left,right query_layer=1
  
# Now use the left and right cat value to separate the inner and
outer boundaries
  GRASS 7.4.0 (nc_basic_spm_grass7):~ > v.extract
my_geol2 layer=2 out=inner_bndry type=boundary
where="left<>-1 and right<>-1"
  GRASS 7.4.0 (nc_basic_spm_grass7):~ > v.extract
my_geol2 layer=2 out=outer_bndry type=boundary where="left=-1 or
right=-1"
  
# Convert inner boundaries to lines for smoothing, and run
generalize
  GRASS 7.4.0 (nc_basic_spm_grass7):~ > v.type
input=inner_bndry out=inner_lines layer=2 from=boundary to=line
  GRASS 7.4.0 (nc_basic_spm_grass7):~ > v.generalize
inner_lines method=douglas output=inner_lines_smooth thresh=1000
--o
  
# Convert back to boundary and merge back with the outer
boundary and recreate area centroids
  GRASS 7.4.0 (nc_basic_spm_grass7):~ > v.type
inner_lines_smooth out=inner_bndry_smooth from=line to=boundary
layer=2 --o
  GRASS 7.4.0 (nc_basic_spm_grass7):~ > v.patch
input=outer_bndry,inner_bndry_smooth output=merge_smooth --o
GRASS 7.4.0 (nc_basic_spm_grass7):~ > v.centroids
merge_smooth output=merged_areas layer=2 option=add

  
You probably will have to do some manual cleaning of the result,
since the smoothed inner boundaries might no longer exactly
intersect with the untouched outer boundary due to overshoots, etc.

If you try this, let us know how it went.

Regards, Micha



On 08/24/2018 06:43 PM, Daniel
  McInerney wrote:


  Hi List,

I'm using v.generalize in a script (excerpt included below) to smooth
the boundaries of a polygon vector dataset. In the attached example
(input_vector.png), I would like to *only* generalize the internal lines
(blue), and leave the external boundary (green line) unchanged. I
thought that the flag, '-l' might provide this functionality, but
unfortunately the resulting external boundary is changed significantly
(attached vgeneralize_output.png).

Is this functionality available in v.generalize or is there another way
that this could be achieved in GRASS?

Thanks in advance.

Best regards,
Daniel.

##run generalisation (step: 1 - douglas)
gscript.run_command('v.generalize', flags='l', overwrite=True,
input='segments', output='segments_douglas', method='douglas', threshold=1)

##run generalisation (step: 2 - sliding average)
gscript.run_command('v.generalize', flags='l', overwrite=True,
input='segments_douglas', output='segments_douglas_slide',
method='sliding_averaging', threshold='2', look_ahead='9', slide='0.1',
iterations='3')

##run generalisation (step: 3 - snake)
gscript.run_command('v.generalize', flags='l', overwrite=True,
input='segments_douglas_slide', output='segments_douglas_slide_snake',
method='snake', threshold='3', alpha='1', beta='1')
 
 

  
  
  
  ___
grass-user mailing list
grass-user@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-user


-- 
Micha Silver
Ben Gurion Univ.
Sde Boker, Remote Sensing Lab
cell: +972-523-665918
  

___
grass-user mailing list
grass-user@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] v.generalize failing on some geoms

[Markus Metz]

On Tue, Feb 13, 2018 at 9:02 AM, Paolo Cavallini 
wrote:
>
> Hi Markus,
>
> Il 12/02/2018 21:46, Markus Metz ha scritto:
>
> > The original data in EPSG:3003 and EPSG:3857 are not identical. The
> > original data in EPSG:3003 are topologically correct, while the original
> > data in EPSG:3857 are not.
>
> interesting. so reprojecting creates errors (tiny gaps, in fact). just
> to document, the process was importing a shapefile into PostGIS, at the
> same time reprojecting from 3003 to 3857

This is the reason why neighboring polygons no longer match each other
exactly.

>
> > Further on, the original data in EPSG:3857
> > contain features that are not present in the original data in EPSG:3003
> > which is quite strange.
>
> forget about this, polygons added after import
>
> > Reprojecting the original data in EPSG:3003 to EPSG:3857 within GRASS
> > works fine, also with subsequent v.generalize.
> >
> > That means that the original data in EPSG:3857 are some reprojected
> > version of the original original data (which are these?). The
> > reprojection step, apparently performed on polygons, not GRASS areas
> > (how did you reproject?), introduced topological errors. Please use
> > native GRASS v.in.ogr + v.proj to reproject polygons.
>
> right; an easy alternative when working with QGIS Processing is to add a
> snap param, which cures the small gaps.

v.in.ogr suggests a range of snapping values specific for the data to be
imported if something goes wrong. In this case snap=0.0001 helps.

Markus M
>
> All the best, and thanks again.
>
> --
> Paolo Cavallini - www.faunalia.eu
> QGIS & PostGIS courses: http://www.faunalia.eu/training.html
> https://www.google.com/trends/explore?date=all=IT=qgis,arcgis
___
grass-user mailing list
grass-user@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] v.generalize failing on some geoms

[Paolo Cavallini]

Hi Markus,

Il 12/02/2018 21:46, Markus Metz ha scritto:

> The original data in EPSG:3003 and EPSG:3857 are not identical. The
> original data in EPSG:3003 are topologically correct, while the original
> data in EPSG:3857 are not.

interesting. so reprojecting creates errors (tiny gaps, in fact). just
to document, the process was importing a shapefile into PostGIS, at the
same time reprojecting from 3003 to 3857

> Further on, the original data in EPSG:3857
> contain features that are not present in the original data in EPSG:3003
> which is quite strange.

forget about this, polygons added after import

> Reprojecting the original data in EPSG:3003 to EPSG:3857 within GRASS
> works fine, also with subsequent v.generalize.
> 
> That means that the original data in EPSG:3857 are some reprojected
> version of the original original data (which are these?). The
> reprojection step, apparently performed on polygons, not GRASS areas
> (how did you reproject?), introduced topological errors. Please use
> native GRASS v.in.ogr + v.proj to reproject polygons.

right; an easy alternative when working with QGIS Processing is to add a
snap param, which cures the small gaps.

All the best, and thanks again.

-- 
Paolo Cavallini - www.faunalia.eu
QGIS & PostGIS courses: http://www.faunalia.eu/training.html
https://www.google.com/trends/explore?date=all=IT=qgis,arcgis
___
grass-user mailing list
grass-user@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] v.generalize failing on some geoms

[Markus Metz]

On Mon, Feb 12, 2018 at 6:22 PM, Paolo Cavallini 
wrote:
>
> Il 11/02/2018 18:34, Markus Metz ha scritto:
>
> > This implies that the data have been reprojected at some stage somewhere
> > from one SRS to the other. The most likely explanation for the different
> > results, particularly the topological errors reported by v.in.ogr, is
> > that non-topological polygons were reprojected. Can you make these geoms
> > in EPSG:3857 and EPSG:3003 available for testing?
>
> Hi Markus,
> thanks for your thoughts. Here the data, original and simplified, in two
> CRS (3003 is simplified correctly, 3857 not):
> http://www.faunalia.eu/~paolo/test_generalize.tar.xz

The original data in EPSG:3003 and EPSG:3857 are not identical. The
original data in EPSG:3003 are topologically correct, while the original
data in EPSG:3857 are not. Further on, the original data in EPSG:3857
contain features that are not present in the original data in EPSG:3003
which is quite strange.

Reprojecting the original data in EPSG:3003 to EPSG:3857 within GRASS works
fine, also with subsequent v.generalize.

That means that the original data in EPSG:3857 are some reprojected version
of the original original data (which are these?). The reprojection step,
apparently performed on polygons, not GRASS areas (how did you reproject?),
introduced topological errors. Please use native GRASS v.in.ogr + v.proj to
reproject polygons.

Markus M

> All the best.
> --
> Paolo Cavallini - www.faunalia.eu
> QGIS & PostGIS courses: http://www.faunalia.eu/training.html
> https://www.google.com/trends/explore?date=all=IT=qgis,arcgis
___
grass-user mailing list
grass-user@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] v.generalize failing on some geoms

[Paolo Cavallini]

Il 11/02/2018 18:34, Markus Metz ha scritto:

> This implies that the data have been reprojected at some stage somewhere
> from one SRS to the other. The most likely explanation for the different
> results, particularly the topological errors reported by v.in.ogr, is
> that non-topological polygons were reprojected. Can you make these geoms
> in EPSG:3857 and EPSG:3003 available for testing?

Hi Markus,
thanks for your thoughts. Here the data, original and simplified, in two
CRS (3003 is simplified correctly, 3857 not):
http://www.faunalia.eu/~paolo/test_generalize.tar.xz
All the best.
-- 
Paolo Cavallini - www.faunalia.eu
QGIS & PostGIS courses: http://www.faunalia.eu/training.html
https://www.google.com/trends/explore?date=all=IT=qgis,arcgis
___
grass-user mailing list
grass-user@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] v.generalize failing on some geoms

[Helmut Kudrnovsky]

>v.generalize is failing to simplify some of my geometries when in
>EPSG:3857. Same geoms in EPSG:3003 were simplified correctly.
>Incorrect borders were reported, unclear to me why.

In your first report you're talking about the same data set in different
EPSG defined projections.

>I'm not explicitly reprojecting it. I tried the command from within 
>QGIS>Processing. Here the relevant commands: 

what is the original projection of your data?



-
best regards
Helmut
--
Sent from: http://osgeo-org.1560.x6.nabble.com/Grass-Users-f3884509.html
___
grass-user mailing list
grass-user@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] v.generalize failing on some geoms

[Paolo Cavallini]

Thanks Markus for the hint.
Replies below.

Il 10/02/2018 22:53, Markus Metz ha scritto:
> This seems to be a reprojection problem.
> 
> If you reprojected the vector data in GRASS with v.in.ogr + v.proj, this
> is a problem of GRASS, granted that v.in.ogr did not complain about
> vector topology.
> 
> If you reprojected non-topological polygons, this is a common problem
> when reprojecting non-topological polygons. Please import these polygons
> first into GRASS, then reproject within GRASS and run v.generalize on
> the reprojected GRASS vector.

I'm not explicitly reprojecting it. I tried the command from within
QGIS>Processing. Here the relevant commands:

g.proj -c proj4="+proj=merc +a=6378137 +b=6378137 +lat_ts=0.0 +lon_0=0.0
+x_0=0.0 +y_0=0 +k=1.0 +units=m +nadgrids=@null +wktext +no_defs"
v.in.ogr min_area=0.0001 snap=-1
input="/tmp/processing63c3c94ab569415fba39f41c12e3e48f"
layer=1518196447.565 output=tmp1518196448046 --overwrite -o
g.region n=5538851.39594 s=5150937.66442 e=1377191.37538 w=1004373.60434
res=100
v.generalize input="tmp1518196448046" method=douglas threshold="1000"
look_ahead="7" reduction="50" -l
output="output081005b4fafd473bbfa438a964de80f1" --overwrite

All the best.
-- 
Paolo Cavallini - www.faunalia.eu
QGIS & PostGIS courses: http://www.faunalia.eu/training.html
https://www.google.com/trends/explore?date=all=IT=qgis,arcgis
___
grass-user mailing list
grass-user@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] v.generalize failing on some geoms

[Paolo Cavallini]

Il 09/02/2018 19:25, Markus Neteler ha scritto:
> On Fri, Feb 9, 2018 at 6:19 PM, Paolo Cavallini  wrote:
>> Hi all,
>> v.generalize is failing to simplify some of my geometries when in
>> EPSG:3857. Same geoms in EPSG:3003 were simplified correctly.
>> Incorrect borders were reported, unclear to me why.
> 
> Please let us know which GRASS GIS version you use.

7.4.0-1
thanks
-- 
Paolo Cavallini - www.faunalia.eu
QGIS & PostGIS courses: http://www.faunalia.eu/training.html
https://www.google.com/trends/explore?date=all=IT=qgis,arcgis
___
grass-user mailing list
grass-user@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] v.generalize failing on some geoms

[Markus Metz]

On Fri, Feb 9, 2018 at 6:19 PM, Paolo Cavallini 
wrote:
>
> Hi all,
> v.generalize is failing to simplify some of my geometries when in
> EPSG:3857. Same geoms in EPSG:3003 were simplified correctly.
> Incorrect borders were reported, unclear to me why.
> All the best, and thanks for any hint.

This seems to be a reprojection problem.

If you reprojected the vector data in GRASS with v.in.ogr + v.proj, this is
a problem of GRASS, granted that v.in.ogr did not complain about vector
topology.

If you reprojected non-topological polygons, this is a common problem when
reprojecting non-topological polygons. Please import these polygons first
into GRASS, then reproject within GRASS and run v.generalize on the
reprojected GRASS vector.

HTH,

Markus M
___
grass-user mailing list
grass-user@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] v.generalize failing on some geoms

[Markus Neteler]

On Fri, Feb 9, 2018 at 6:19 PM, Paolo Cavallini  wrote:
> Hi all,
> v.generalize is failing to simplify some of my geometries when in
> EPSG:3857. Same geoms in EPSG:3003 were simplified correctly.
> Incorrect borders were reported, unclear to me why.

Please let us know which GRASS GIS version you use.

Thanks
Markus
___
grass-user mailing list
grass-user@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] v.generalize crash

[Paolo Cavallini]

Il 22/02/2016 21:33, Markus Neteler ha scritto:

> Sure - here is our release schedule:
> https://trac.osgeo.org/grass/milestone/7.0.4
> 
> In some weeks we'll have the first RC. Some more known bugs to be
> eliminated since then!

ok, thanks.
-- 
Paolo Cavallini - www.faunalia.eu
QGIS & PostGIS courses: http://www.faunalia.eu/training.html
___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] v.generalize crash

[Markus Neteler]

On Mon, Feb 22, 2016 at 8:23 AM, Paolo Cavallini  wrote:
> Il 21/02/2016 21:10, Markus Neteler ha scritto:
>> On Sat, Feb 20, 2016 at 8:50 AM, Paolo Cavallini  
>> wrote:
>> ...
 I have opened a ticket:
 https://trac.osgeo.org/grass/ticket/2929
>>
>> It got fixed in 7.0.svn - please try (will go into 7.0.4).
>
> Hi Markus,
> that was fast, thanks. When is 7.0.4 scheduled? Given the crash, I
> believe having fixed packages soon is highly desirable.

Sure - here is our release schedule:
https://trac.osgeo.org/grass/milestone/7.0.4

In some weeks we'll have the first RC. Some more known bugs to be
eliminated since then!

best,
Markus
___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] v.generalize crash

[Paolo Cavallini]

Il 21/02/2016 21:10, Markus Neteler ha scritto:
> On Sat, Feb 20, 2016 at 8:50 AM, Paolo Cavallini  
> wrote:
> ...
>>> I have opened a ticket:
>>> https://trac.osgeo.org/grass/ticket/2929
> 
> It got fixed in 7.0.svn - please try (will go into 7.0.4).

Hi Markus,
that was fast, thanks. When is 7.0.4 scheduled? Given the crash, I
believe having fixed packages soon is highly desirable.
All the best.

-- 
Paolo Cavallini - www.faunalia.eu
QGIS & PostGIS courses: http://www.faunalia.eu/training.html
___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] v.generalize crash

[Markus Neteler]

On Sat, Feb 20, 2016 at 8:50 AM, Paolo Cavallini  wrote:
...
>> I have opened a ticket:
>> https://trac.osgeo.org/grass/ticket/2929

It got fixed in 7.0.svn - please try (will go into 7.0.4).

Markus
___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] v.generalize crash

[Paolo Cavallini]

Thanks Markus.
All the best.

Il 20 febbraio 2016 08:19:14 CET, Markus Neteler  ha scritto:
>Hi Paolo,
>
>On Mon, Feb 15, 2016 at 4:38 PM, Paolo Cavallini
> wrote:
>> Hi all,
>> I get consistent crashes of v.generalize on Debian sid, official deb,
>e.g.:
>>
>> GRASS 7.0.3 (world):~ > v.generalize input=corine@PERMANENT layer=1
>> type=area type=line,boundary,area method=douglas threshold=5.0
>> look_ahead=7 reduction=50 slide=0.5 angle_thresh=3 degree_thresh=0
>> closeness_thresh=0 betweeness_thresh=0 alpha=1.0 beta=1.0
>iterations=1
>> output=corine_simpl
>
>Same issue here:
>
>GRASS 7.0.3svn (nc_spm_08_grass7):~ > v.generalize input=soils_general
>layer=1 type=area type=line,boundary,area method=douglas threshold=5.0
>look_ahead=7 reduction=50 slide=0.5 angle_thresh=3 degree_thresh=0
>closeness_thresh=0 betweeness_thresh=0 alpha=1.0 beta=1.0 iterations=1
>output=soils_general_simpl
>...
>Number of areas: 1428
>Number of isles: 244
>-
>Generalization (douglas)...
>Using threshold: 5 meters
>Segmentation fault (core dumped)
>
>with "gdb"
>Program received signal SIGSEGV, Segmentation fault.
>0x77bb2146 in Vect_line_intersection2 ()
>from
>/home/neteler/software/grass70/dist.x86_64-pc-linux-gnu/lib/libgrass_vector.7.0.3svn.so
>
>I have opened a ticket:
>https://trac.osgeo.org/grass/ticket/2929
>
>Markus

-- 
Sent from mobile. Sorry for being short
http://faunalia.eu___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] v.generalize crash

[Markus Neteler]

Hi Paolo,

On Mon, Feb 15, 2016 at 4:38 PM, Paolo Cavallini  wrote:
> Hi all,
> I get consistent crashes of v.generalize on Debian sid, official deb, e.g.:
>
> GRASS 7.0.3 (world):~ > v.generalize input=corine@PERMANENT layer=1
> type=area type=line,boundary,area method=douglas threshold=5.0
> look_ahead=7 reduction=50 slide=0.5 angle_thresh=3 degree_thresh=0
> closeness_thresh=0 betweeness_thresh=0 alpha=1.0 beta=1.0 iterations=1
> output=corine_simpl

Same issue here:

GRASS 7.0.3svn (nc_spm_08_grass7):~ > v.generalize input=soils_general
layer=1 type=area type=line,boundary,area method=douglas threshold=5.0
look_ahead=7 reduction=50 slide=0.5 angle_thresh=3 degree_thresh=0
closeness_thresh=0 betweeness_thresh=0 alpha=1.0 beta=1.0 iterations=1
output=soils_general_simpl
...
Number of areas: 1428
Number of isles: 244
-
Generalization (douglas)...
Using threshold: 5 meters
Segmentation fault (core dumped)

with "gdb"
Program received signal SIGSEGV, Segmentation fault.
0x77bb2146 in Vect_line_intersection2 ()
   from 
/home/neteler/software/grass70/dist.x86_64-pc-linux-gnu/lib/libgrass_vector.7.0.3svn.so

I have opened a ticket:
https://trac.osgeo.org/grass/ticket/2929

Markus
___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] v.generalize: does it take forever?

[Vaclav Petras]

On Mon, Feb 9, 2015 at 4:52 PM, Fábio Dias fabio.d...@gmail.com wrote:

 I switched to postgis for data storage and the v.generalize time went
 down to 130ish minutes, all processes working in parallel.

 I'm happy now :) thanks you guys very much.


Thanks for reporting this back. What about a blog post, or something like
that, on this topic? I believe there is a lot of people interested in some
benchmarks.

Vaclav

-=--=-=-
 Fábio Augusto Salve Dias
 ICMC - USP
 http://sites.google.com/site/fabiodias/


 On Tue, Jan 27, 2015 at 8:50 PM, Fábio Dias fabio.d...@gmail.com wrote:
  Hi,
 
  I've kept an iotop, cumulative, running while the generalization run.
  No disk IO involved, just a couple of postgre stats. I believe the OS
  is keeping everything in RAM cache. I don't believe the disk is a
  bottleneck either, it is a 10 disk raid of 15k rpm disks, it's really
  fast.
 
  I interrupted the processing, moved everything into postgres and
  started over. I'm still loading the shapefiles (that I'm doing one at
  a time), I'll start the 15 processes as soon as it is loaded. As soon
  as that stabilizes, I'll report back.
 
 
  On a related note, wouldn't it be interesting to always try to
  simplify a line? As I understood the code, if the simplification fails
  for any reason, the original line is used. Might be interesting to
  have an option that makes the algorithm retry that line, with half the
  threshold, for instance. It's kind of weird for me to see one side of
  something really simplified while the other side really complicated :)
 
  F
  -=--=-=-
  Fábio Augusto Salve Dias
  ICMC - USP
  http://sites.google.com/site/fabiodias/
 
 
  On Tue, Jan 27, 2015 at 7:56 PM, Markus Metz
  markus.metz.gisw...@gmail.com wrote:
  On Mon, Jan 26, 2015 at 3:54 PM, Fábio Dias fabio.d...@gmail.com
 wrote:
  Hi,
 
  The machine has 128Gb of ram. Doesn't matter what I do, I can't make a
  dent on it. Even with everything cached in ram (shp files, database,
  the whole lot), there is still free memory.
 
  OK, it's not RAM.
 
 
  I'm asking about the database because the behavior I'm seeing on 'top'
  looks like the one you get when mutexes are involved. The processes
  don't go all to 100% processing at same time (and the machine has 64
  processors, so no dent there either), except for the v.in.ogr.
 
  The v.generailze processes should be at 100% while generalizing,
  unless the disk can not keep up with multiple simultaneous IO
  requests. The tables are copied only after the generalization finished
  (100% reached).
 
  What it
  looks like is that something is locking each process and they are
  taking turns. Considering how 'lite' the sqlite appears to be, and the
  weird locking behavior that was mentioned in other threads (I'm not
  getting the locked message here... I did, when I was running 2
  parallel v.in.ogr), isn't it likely to be the culprit? Should I change
  it to a more 'non-lite' system, like postgres for instance?
 
  That could make sense when running multiple processes in parallel. An
  alternative would be to create a separate mapset for each process and
  at the end copy the results back to the main mapset.
 
  Technically, it is not possible that the new v.generalize version in
  trunk (G71) is slower than the old version as in relbr70 because the
  new version updates only those parts of the topology that actually get
  changed. The old version also updates components that do not get
  changed, this is quite time-consuming.
 
  I understand you like to go for the big nail immediately, but maybe it
  is worth testing first on a smaller sample?
 
  Markus M
 
 
  F
  -=--=-=-
  Fábio Augusto Salve Dias
  ICMC - USP
  http://sites.google.com/site/fabiodias/
 
 
  On Mon, Jan 26, 2015 at 7:22 AM, Markus Metz
  markus.metz.gisw...@gmail.com wrote:
  On Mon, Jan 26, 2015 at 9:30 AM, Markus Metz
  markus.metz.gisw...@gmail.com wrote:
  On Sun, Jan 25, 2015 at 6:11 PM, Fábio Dias fabio.d...@gmail.com
 wrote:
  Hi,
 
  Running r64249, with a couple of stuff in parallel using . It seems
  to be considerably slower.
 
  Very strange. Are you using trunk or GRASS 7.0?
 
  Here, v.generalize on a TerraClass tile is down from 25 minutes to 13
 seconds.
 
 
  More than 100h, no 1% printed. To be fair,
  I'm not entirely sure I'll see it when it prints, 10 v.generalize
  running (5 for each year) + 1 v.in.ogr for 2012. That v.in.ogr is
  running for almost 100h too. I'm loading the shps directly, as
 advised
  way, way back in this thread.
 
  What exactly do you mean with loading shps directly? For
  v.generalize, you should import them with v.in.ogr.
 
  What about memory consumption on your system? With 10 v.generalize +
 1
  v.in.ogr on such a big dataset, quite a lot of memory would be used.
 
  Markus M
 
 
  AFAIK, no disk is been used, the whole thing is cached (after more
  than 24h processing, cumulative iotop shows only a few mb
  written/read). I'm no longer using a ramdisk for the grassdata 

Re: [GRASS-user] v.generalize: does it take forever?

[Fábio Dias]

I switched to postgis for data storage and the v.generalize time went
down to 130ish minutes, all processes working in parallel.

I'm happy now :) thanks you guys very much.
-=--=-=-
Fábio Augusto Salve Dias
ICMC - USP
http://sites.google.com/site/fabiodias/


On Tue, Jan 27, 2015 at 8:50 PM, Fábio Dias fabio.d...@gmail.com wrote:
 Hi,

 I've kept an iotop, cumulative, running while the generalization run.
 No disk IO involved, just a couple of postgre stats. I believe the OS
 is keeping everything in RAM cache. I don't believe the disk is a
 bottleneck either, it is a 10 disk raid of 15k rpm disks, it's really
 fast.

 I interrupted the processing, moved everything into postgres and
 started over. I'm still loading the shapefiles (that I'm doing one at
 a time), I'll start the 15 processes as soon as it is loaded. As soon
 as that stabilizes, I'll report back.


 On a related note, wouldn't it be interesting to always try to
 simplify a line? As I understood the code, if the simplification fails
 for any reason, the original line is used. Might be interesting to
 have an option that makes the algorithm retry that line, with half the
 threshold, for instance. It's kind of weird for me to see one side of
 something really simplified while the other side really complicated :)

 F
 -=--=-=-
 Fábio Augusto Salve Dias
 ICMC - USP
 http://sites.google.com/site/fabiodias/


 On Tue, Jan 27, 2015 at 7:56 PM, Markus Metz
 markus.metz.gisw...@gmail.com wrote:
 On Mon, Jan 26, 2015 at 3:54 PM, Fábio Dias fabio.d...@gmail.com wrote:
 Hi,

 The machine has 128Gb of ram. Doesn't matter what I do, I can't make a
 dent on it. Even with everything cached in ram (shp files, database,
 the whole lot), there is still free memory.

 OK, it's not RAM.


 I'm asking about the database because the behavior I'm seeing on 'top'
 looks like the one you get when mutexes are involved. The processes
 don't go all to 100% processing at same time (and the machine has 64
 processors, so no dent there either), except for the v.in.ogr.

 The v.generailze processes should be at 100% while generalizing,
 unless the disk can not keep up with multiple simultaneous IO
 requests. The tables are copied only after the generalization finished
 (100% reached).

 What it
 looks like is that something is locking each process and they are
 taking turns. Considering how 'lite' the sqlite appears to be, and the
 weird locking behavior that was mentioned in other threads (I'm not
 getting the locked message here... I did, when I was running 2
 parallel v.in.ogr), isn't it likely to be the culprit? Should I change
 it to a more 'non-lite' system, like postgres for instance?

 That could make sense when running multiple processes in parallel. An
 alternative would be to create a separate mapset for each process and
 at the end copy the results back to the main mapset.

 Technically, it is not possible that the new v.generalize version in
 trunk (G71) is slower than the old version as in relbr70 because the
 new version updates only those parts of the topology that actually get
 changed. The old version also updates components that do not get
 changed, this is quite time-consuming.

 I understand you like to go for the big nail immediately, but maybe it
 is worth testing first on a smaller sample?

 Markus M


 F
 -=--=-=-
 Fábio Augusto Salve Dias
 ICMC - USP
 http://sites.google.com/site/fabiodias/


 On Mon, Jan 26, 2015 at 7:22 AM, Markus Metz
 markus.metz.gisw...@gmail.com wrote:
 On Mon, Jan 26, 2015 at 9:30 AM, Markus Metz
 markus.metz.gisw...@gmail.com wrote:
 On Sun, Jan 25, 2015 at 6:11 PM, Fábio Dias fabio.d...@gmail.com wrote:
 Hi,

 Running r64249, with a couple of stuff in parallel using . It seems
 to be considerably slower.

 Very strange. Are you using trunk or GRASS 7.0?

 Here, v.generalize on a TerraClass tile is down from 25 minutes to 13 
 seconds.


 More than 100h, no 1% printed. To be fair,
 I'm not entirely sure I'll see it when it prints, 10 v.generalize
 running (5 for each year) + 1 v.in.ogr for 2012. That v.in.ogr is
 running for almost 100h too. I'm loading the shps directly, as advised
 way, way back in this thread.

 What exactly do you mean with loading shps directly? For
 v.generalize, you should import them with v.in.ogr.

 What about memory consumption on your system? With 10 v.generalize + 1
 v.in.ogr on such a big dataset, quite a lot of memory would be used.

 Markus M


 AFAIK, no disk is been used, the whole thing is cached (after more
 than 24h processing, cumulative iotop shows only a few mb
 written/read). I'm no longer using a ramdisk for the grassdata dir.

 However, it appears to be considerably slower, probably because of the
 parallel running jobs.

 My question then would be, considering the thread I saw about sqlite,
 should I be using something else as backend? When it starts to make
 sense to change it?

 F

 -=--=-=-
 Fábio Augusto Salve Dias
 ICMC - USP
 http://sites.google.com/site/fabiodias/


 On 

Re: [GRASS-user] v.generalize: does it take forever?

[Markus Metz]

On Mon, Jan 26, 2015 at 3:54 PM, Fábio Dias fabio.d...@gmail.com wrote:
 Hi,

 The machine has 128Gb of ram. Doesn't matter what I do, I can't make a
 dent on it. Even with everything cached in ram (shp files, database,
 the whole lot), there is still free memory.

OK, it's not RAM.


 I'm asking about the database because the behavior I'm seeing on 'top'
 looks like the one you get when mutexes are involved. The processes
 don't go all to 100% processing at same time (and the machine has 64
 processors, so no dent there either), except for the v.in.ogr.

The v.generailze processes should be at 100% while generalizing,
unless the disk can not keep up with multiple simultaneous IO
requests. The tables are copied only after the generalization finished
(100% reached).

 What it
 looks like is that something is locking each process and they are
 taking turns. Considering how 'lite' the sqlite appears to be, and the
 weird locking behavior that was mentioned in other threads (I'm not
 getting the locked message here... I did, when I was running 2
 parallel v.in.ogr), isn't it likely to be the culprit? Should I change
 it to a more 'non-lite' system, like postgres for instance?

That could make sense when running multiple processes in parallel. An
alternative would be to create a separate mapset for each process and
at the end copy the results back to the main mapset.

Technically, it is not possible that the new v.generalize version in
trunk (G71) is slower than the old version as in relbr70 because the
new version updates only those parts of the topology that actually get
changed. The old version also updates components that do not get
changed, this is quite time-consuming.

I understand you like to go for the big nail immediately, but maybe it
is worth testing first on a smaller sample?

Markus M


 F
 -=--=-=-
 Fábio Augusto Salve Dias
 ICMC - USP
 http://sites.google.com/site/fabiodias/


 On Mon, Jan 26, 2015 at 7:22 AM, Markus Metz
 markus.metz.gisw...@gmail.com wrote:
 On Mon, Jan 26, 2015 at 9:30 AM, Markus Metz
 markus.metz.gisw...@gmail.com wrote:
 On Sun, Jan 25, 2015 at 6:11 PM, Fábio Dias fabio.d...@gmail.com wrote:
 Hi,

 Running r64249, with a couple of stuff in parallel using . It seems
 to be considerably slower.

 Very strange. Are you using trunk or GRASS 7.0?

 Here, v.generalize on a TerraClass tile is down from 25 minutes to 13 
 seconds.


 More than 100h, no 1% printed. To be fair,
 I'm not entirely sure I'll see it when it prints, 10 v.generalize
 running (5 for each year) + 1 v.in.ogr for 2012. That v.in.ogr is
 running for almost 100h too. I'm loading the shps directly, as advised
 way, way back in this thread.

 What exactly do you mean with loading shps directly? For
 v.generalize, you should import them with v.in.ogr.

 What about memory consumption on your system? With 10 v.generalize + 1
 v.in.ogr on such a big dataset, quite a lot of memory would be used.

 Markus M


 AFAIK, no disk is been used, the whole thing is cached (after more
 than 24h processing, cumulative iotop shows only a few mb
 written/read). I'm no longer using a ramdisk for the grassdata dir.

 However, it appears to be considerably slower, probably because of the
 parallel running jobs.

 My question then would be, considering the thread I saw about sqlite,
 should I be using something else as backend? When it starts to make
 sense to change it?

 F

 -=--=-=-
 Fábio Augusto Salve Dias
 ICMC - USP
 http://sites.google.com/site/fabiodias/


 On Wed, Jan 14, 2015 at 1:06 PM, Markus Neteler nete...@osgeo.org wrote:
 On Wed, Jan 14, 2015 at 3:54 PM, Fábio Dias fabio.d...@gmail.com wrote:
 ...
 What would be the best way to do that in parallel? One mapset for each
 year? Can I run multiple v.generalizes on the same input with
 different outputs?

 Yes sure.

 My first thought was to run completely separated grass processes for
 each simplification, but I didn't find a way to make it search
 something different than .grass / .grass70 for the configuration
 stuff

 Maybe take a look at this approach
 http://grasswiki.osgeo.org/wiki/Parallel_GRASS_jobs#Grid_Engine

 but even sending different v.generalize jobs to background () should
 work if you have enough RAM.

 markusN
___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] v.generalize: does it take forever?

[Fábio Dias]

Hi,

I've kept an iotop, cumulative, running while the generalization run.
No disk IO involved, just a couple of postgre stats. I believe the OS
is keeping everything in RAM cache. I don't believe the disk is a
bottleneck either, it is a 10 disk raid of 15k rpm disks, it's really
fast.

I interrupted the processing, moved everything into postgres and
started over. I'm still loading the shapefiles (that I'm doing one at
a time), I'll start the 15 processes as soon as it is loaded. As soon
as that stabilizes, I'll report back.


On a related note, wouldn't it be interesting to always try to
simplify a line? As I understood the code, if the simplification fails
for any reason, the original line is used. Might be interesting to
have an option that makes the algorithm retry that line, with half the
threshold, for instance. It's kind of weird for me to see one side of
something really simplified while the other side really complicated :)

F
-=--=-=-
Fábio Augusto Salve Dias
ICMC - USP
http://sites.google.com/site/fabiodias/


On Tue, Jan 27, 2015 at 7:56 PM, Markus Metz
markus.metz.gisw...@gmail.com wrote:
 On Mon, Jan 26, 2015 at 3:54 PM, Fábio Dias fabio.d...@gmail.com wrote:
 Hi,

 The machine has 128Gb of ram. Doesn't matter what I do, I can't make a
 dent on it. Even with everything cached in ram (shp files, database,
 the whole lot), there is still free memory.

 OK, it's not RAM.


 I'm asking about the database because the behavior I'm seeing on 'top'
 looks like the one you get when mutexes are involved. The processes
 don't go all to 100% processing at same time (and the machine has 64
 processors, so no dent there either), except for the v.in.ogr.

 The v.generailze processes should be at 100% while generalizing,
 unless the disk can not keep up with multiple simultaneous IO
 requests. The tables are copied only after the generalization finished
 (100% reached).

 What it
 looks like is that something is locking each process and they are
 taking turns. Considering how 'lite' the sqlite appears to be, and the
 weird locking behavior that was mentioned in other threads (I'm not
 getting the locked message here... I did, when I was running 2
 parallel v.in.ogr), isn't it likely to be the culprit? Should I change
 it to a more 'non-lite' system, like postgres for instance?

 That could make sense when running multiple processes in parallel. An
 alternative would be to create a separate mapset for each process and
 at the end copy the results back to the main mapset.

 Technically, it is not possible that the new v.generalize version in
 trunk (G71) is slower than the old version as in relbr70 because the
 new version updates only those parts of the topology that actually get
 changed. The old version also updates components that do not get
 changed, this is quite time-consuming.

 I understand you like to go for the big nail immediately, but maybe it
 is worth testing first on a smaller sample?

 Markus M


 F
 -=--=-=-
 Fábio Augusto Salve Dias
 ICMC - USP
 http://sites.google.com/site/fabiodias/


 On Mon, Jan 26, 2015 at 7:22 AM, Markus Metz
 markus.metz.gisw...@gmail.com wrote:
 On Mon, Jan 26, 2015 at 9:30 AM, Markus Metz
 markus.metz.gisw...@gmail.com wrote:
 On Sun, Jan 25, 2015 at 6:11 PM, Fábio Dias fabio.d...@gmail.com wrote:
 Hi,

 Running r64249, with a couple of stuff in parallel using . It seems
 to be considerably slower.

 Very strange. Are you using trunk or GRASS 7.0?

 Here, v.generalize on a TerraClass tile is down from 25 minutes to 13 
 seconds.


 More than 100h, no 1% printed. To be fair,
 I'm not entirely sure I'll see it when it prints, 10 v.generalize
 running (5 for each year) + 1 v.in.ogr for 2012. That v.in.ogr is
 running for almost 100h too. I'm loading the shps directly, as advised
 way, way back in this thread.

 What exactly do you mean with loading shps directly? For
 v.generalize, you should import them with v.in.ogr.

 What about memory consumption on your system? With 10 v.generalize + 1
 v.in.ogr on such a big dataset, quite a lot of memory would be used.

 Markus M


 AFAIK, no disk is been used, the whole thing is cached (after more
 than 24h processing, cumulative iotop shows only a few mb
 written/read). I'm no longer using a ramdisk for the grassdata dir.

 However, it appears to be considerably slower, probably because of the
 parallel running jobs.

 My question then would be, considering the thread I saw about sqlite,
 should I be using something else as backend? When it starts to make
 sense to change it?

 F

 -=--=-=-
 Fábio Augusto Salve Dias
 ICMC - USP
 http://sites.google.com/site/fabiodias/


 On Wed, Jan 14, 2015 at 1:06 PM, Markus Neteler nete...@osgeo.org wrote:
 On Wed, Jan 14, 2015 at 3:54 PM, Fábio Dias fabio.d...@gmail.com wrote:
 ...
 What would be the best way to do that in parallel? One mapset for each
 year? Can I run multiple v.generalizes on the same input with
 different outputs?

 Yes sure.

 My first thought was to run completely 

Re: [GRASS-user] v.generalize: does it take forever?

[Fábio Dias]

Hi,

Running trunk. I was loading the shps into postgis then importing them
to grass. Now I'm importing the shps directly.

The machine has 128Gb of ram. Doesn't matter what I do, I can't make a
dent on it. Even with everything cached in ram (shp files, database,
the whole lot), there is still free memory.

I'm asking about the database because the behavior I'm seeing on 'top'
looks like the one you get when mutexes are involved. The processes
don't go all to 100% processing at same time (and the machine has 64
processors, so no dent there either), except for the v.in.ogr. What it
looks like is that something is locking each process and they are
taking turns. Considering how 'lite' the sqlite appears to be, and the
weird locking behavior that was mentioned in other threads (I'm not
getting the locked message here... I did, when I was running 2
parallel v.in.ogr), isn't it likely to be the culprit? Should I change
it to a more 'non-lite' system, like postgres for instance?

F
-=--=-=-
Fábio Augusto Salve Dias
ICMC - USP
http://sites.google.com/site/fabiodias/


On Mon, Jan 26, 2015 at 7:22 AM, Markus Metz
markus.metz.gisw...@gmail.com wrote:
 On Mon, Jan 26, 2015 at 9:30 AM, Markus Metz
 markus.metz.gisw...@gmail.com wrote:
 On Sun, Jan 25, 2015 at 6:11 PM, Fábio Dias fabio.d...@gmail.com wrote:
 Hi,

 Running r64249, with a couple of stuff in parallel using . It seems
 to be considerably slower.

 Very strange. Are you using trunk or GRASS 7.0?

 Here, v.generalize on a TerraClass tile is down from 25 minutes to 13 seconds.


 More than 100h, no 1% printed. To be fair,
 I'm not entirely sure I'll see it when it prints, 10 v.generalize
 running (5 for each year) + 1 v.in.ogr for 2012. That v.in.ogr is
 running for almost 100h too. I'm loading the shps directly, as advised
 way, way back in this thread.

 What exactly do you mean with loading shps directly? For
 v.generalize, you should import them with v.in.ogr.

 What about memory consumption on your system? With 10 v.generalize + 1
 v.in.ogr on such a big dataset, quite a lot of memory would be used.

 Markus M


 AFAIK, no disk is been used, the whole thing is cached (after more
 than 24h processing, cumulative iotop shows only a few mb
 written/read). I'm no longer using a ramdisk for the grassdata dir.

 However, it appears to be considerably slower, probably because of the
 parallel running jobs.

 My question then would be, considering the thread I saw about sqlite,
 should I be using something else as backend? When it starts to make
 sense to change it?

 F

 -=--=-=-
 Fábio Augusto Salve Dias
 ICMC - USP
 http://sites.google.com/site/fabiodias/


 On Wed, Jan 14, 2015 at 1:06 PM, Markus Neteler nete...@osgeo.org wrote:
 On Wed, Jan 14, 2015 at 3:54 PM, Fábio Dias fabio.d...@gmail.com wrote:
 ...
 What would be the best way to do that in parallel? One mapset for each
 year? Can I run multiple v.generalizes on the same input with
 different outputs?

 Yes sure.

 My first thought was to run completely separated grass processes for
 each simplification, but I didn't find a way to make it search
 something different than .grass / .grass70 for the configuration
 stuff

 Maybe take a look at this approach
 http://grasswiki.osgeo.org/wiki/Parallel_GRASS_jobs#Grid_Engine

 but even sending different v.generalize jobs to background () should
 work if you have enough RAM.

 markusN
___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] v.generalize: does it take forever?

[Markus Metz]

On Sun, Jan 25, 2015 at 6:11 PM, Fábio Dias fabio.d...@gmail.com wrote:
 Hi,

 Running r64249, with a couple of stuff in parallel using . It seems
 to be considerably slower.

Very strange. Are you using trunk or GRASS 7.0?

 More than 100h, no 1% printed. To be fair,
 I'm not entirely sure I'll see it when it prints, 10 v.generalize
 running (5 for each year) + 1 v.in.ogr for 2012. That v.in.ogr is
 running for almost 100h too. I'm loading the shps directly, as advised
 way, way back in this thread.

What exactly do you mean with loading shps directly? For
v.generalize, you should import them with v.in.ogr.

What about memory consumption on your system? With 10 v.generalize + 1
v.in.ogr on such a big dataset, quite a lot of memory would be used.

Markus M


 AFAIK, no disk is been used, the whole thing is cached (after more
 than 24h processing, cumulative iotop shows only a few mb
 written/read). I'm no longer using a ramdisk for the grassdata dir.

 However, it appears to be considerably slower, probably because of the
 parallel running jobs.

 My question then would be, considering the thread I saw about sqlite,
 should I be using something else as backend? When it starts to make
 sense to change it?

 F

 -=--=-=-
 Fábio Augusto Salve Dias
 ICMC - USP
 http://sites.google.com/site/fabiodias/


 On Wed, Jan 14, 2015 at 1:06 PM, Markus Neteler nete...@osgeo.org wrote:
 On Wed, Jan 14, 2015 at 3:54 PM, Fábio Dias fabio.d...@gmail.com wrote:
 ...
 What would be the best way to do that in parallel? One mapset for each
 year? Can I run multiple v.generalizes on the same input with
 different outputs?

 Yes sure.

 My first thought was to run completely separated grass processes for
 each simplification, but I didn't find a way to make it search
 something different than .grass / .grass70 for the configuration
 stuff

 Maybe take a look at this approach
 http://grasswiki.osgeo.org/wiki/Parallel_GRASS_jobs#Grid_Engine

 but even sending different v.generalize jobs to background () should
 work if you have enough RAM.

 markusN
___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] v.generalize: does it take forever?

[Markus Metz]

On Mon, Jan 26, 2015 at 9:30 AM, Markus Metz
markus.metz.gisw...@gmail.com wrote:
 On Sun, Jan 25, 2015 at 6:11 PM, Fábio Dias fabio.d...@gmail.com wrote:
 Hi,

 Running r64249, with a couple of stuff in parallel using . It seems
 to be considerably slower.

 Very strange. Are you using trunk or GRASS 7.0?

Here, v.generalize on a TerraClass tile is down from 25 minutes to 13 seconds.


 More than 100h, no 1% printed. To be fair,
 I'm not entirely sure I'll see it when it prints, 10 v.generalize
 running (5 for each year) + 1 v.in.ogr for 2012. That v.in.ogr is
 running for almost 100h too. I'm loading the shps directly, as advised
 way, way back in this thread.

 What exactly do you mean with loading shps directly? For
 v.generalize, you should import them with v.in.ogr.

 What about memory consumption on your system? With 10 v.generalize + 1
 v.in.ogr on such a big dataset, quite a lot of memory would be used.

 Markus M


 AFAIK, no disk is been used, the whole thing is cached (after more
 than 24h processing, cumulative iotop shows only a few mb
 written/read). I'm no longer using a ramdisk for the grassdata dir.

 However, it appears to be considerably slower, probably because of the
 parallel running jobs.

 My question then would be, considering the thread I saw about sqlite,
 should I be using something else as backend? When it starts to make
 sense to change it?

 F

 -=--=-=-
 Fábio Augusto Salve Dias
 ICMC - USP
 http://sites.google.com/site/fabiodias/


 On Wed, Jan 14, 2015 at 1:06 PM, Markus Neteler nete...@osgeo.org wrote:
 On Wed, Jan 14, 2015 at 3:54 PM, Fábio Dias fabio.d...@gmail.com wrote:
 ...
 What would be the best way to do that in parallel? One mapset for each
 year? Can I run multiple v.generalizes on the same input with
 different outputs?

 Yes sure.

 My first thought was to run completely separated grass processes for
 each simplification, but I didn't find a way to make it search
 something different than .grass / .grass70 for the configuration
 stuff

 Maybe take a look at this approach
 http://grasswiki.osgeo.org/wiki/Parallel_GRASS_jobs#Grid_Engine

 but even sending different v.generalize jobs to background () should
 work if you have enough RAM.

 markusN
___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] v.generalize: does it take forever?

[Fábio Dias]

Hi,

Running r64249, with a couple of stuff in parallel using . It seems
to be considerably slower. More than 100h, no 1% printed. To be fair,
I'm not entirely sure I'll see it when it prints, 10 v.generalize
running (5 for each year) + 1 v.in.ogr for 2012. That v.in.ogr is
running for almost 100h too. I'm loading the shps directly, as advised
way, way back in this thread.

AFAIK, no disk is been used, the whole thing is cached (after more
than 24h processing, cumulative iotop shows only a few mb
written/read). I'm no longer using a ramdisk for the grassdata dir.

However, it appears to be considerably slower, probably because of the
parallel running jobs.

My question then would be, considering the thread I saw about sqlite,
should I be using something else as backend? When it starts to make
sense to change it?

F

-=--=-=-
Fábio Augusto Salve Dias
ICMC - USP
http://sites.google.com/site/fabiodias/


On Wed, Jan 14, 2015 at 1:06 PM, Markus Neteler nete...@osgeo.org wrote:
 On Wed, Jan 14, 2015 at 3:54 PM, Fábio Dias fabio.d...@gmail.com wrote:
 ...
 What would be the best way to do that in parallel? One mapset for each
 year? Can I run multiple v.generalizes on the same input with
 different outputs?

 Yes sure.

 My first thought was to run completely separated grass processes for
 each simplification, but I didn't find a way to make it search
 something different than .grass / .grass70 for the configuration
 stuff

 Maybe take a look at this approach
 http://grasswiki.osgeo.org/wiki/Parallel_GRASS_jobs#Grid_Engine

 but even sending different v.generalize jobs to background () should
 work if you have enough RAM.

 markusN
___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] v.generalize: does it take forever?

[Markus Metz]

On Sun, Jan 11, 2015 at 11:32 PM, Markus Metz
markus.metz.gisw...@gmail.com wrote:
 On Sat, Jan 10, 2015 at 7:23 PM, Fábio Dias fabio.d...@gmail.com wrote:
 I have optimized the GRASS vector library in trunk r64032 and added
 another topology check to v.generalize in trunk r64033. The profile of
 v.generalize now shows that it is limited by disk I/O speed (on my
 laptop with a standard laptop-like spinning HDD), which means that the
 algorithms are, under the test conditions, close to their optimum.
 This picture might change as soon as you use a high-performance server
 or a SSD.


 Then I should do a profile on my current setup.

 I have updated v.generalize again in trunk r64067. Please test the
 latest version.


 [...] the Terraclass
 shapefiles are full of errors. If you want to fix these errors, this
 will take some time.

 You know this dataset? The errors are really bugging me. It is, mostly
 due to the process/tools they usually use. We have passed over the
 request for a more topologically correct approach. Maybe on the next
 iteration. But I'll create another thread asking advice regarding
 these errors shortly :)

 I know the Terraclass dataset a bit. I used some tiles for testing. I
 was not able to import any of my test tiles without errors (after
 years of thinking about the conversion of non-topological vectors to
 topological vectors). Terraclass data are based on PRODES data, which
 I know pretty well. The PRODES classification also comes as shapfiles
 which are also full of errors, but these I managed to remove by
 carefully choosing the snapping threshold for v.in.ogr.

 By not previously dissolving and further doing v.clean tool=break the
 original data, I've reduced the processing time from more than 30h for
 1% to 24h to 11%. With the latest release, 9% in 18h.

 9% in 18h seems promising.

As of trunk r64234, the simplification itself should be done within
minutes (heavy optimization, only updating those parts of the vector
topology that actually get changed). Please test.

Markus M



 However, this whole thing got me thinking about you said on an early message:

 The check_topo function can not be executed in parallel because 1)
 topology must not be modified for several boundaries in parallel, 2)
 data are written to disk, and disk IO is by nature not parallel.

 Well, disk IO, there's not much we can do about it.

 We can here and there sometimes reduce disk IO (which I did in some of
 my recent changes).

 Markus M
___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] v.generalize: does it take forever?

[Fábio Dias]

Hello,

Hopefully my last question regarding v.generalize and speeding up the process.

Context:

I have multiple years of data that need to be generalized. For each
year, I need a number of different generalizations (specific number
TBD).

Question:

What would be the best way to do that in parallel? One mapset for each
year? Can I run multiple v.generalizes on the same input with
different outputs?

My first thought was to run completely separated grass processes for
each simplification, but I didn't find a way to make it search
something different than .grass / .grass70 for the configuration
stuff

Thanks again

F
-=--=-=-
Fábio Augusto Salve Dias
ICMC - USP
http://sites.google.com/site/fabiodias/


On Sun, Jan 11, 2015 at 8:32 PM, Markus Metz
markus.metz.gisw...@gmail.com wrote:
 On Sat, Jan 10, 2015 at 7:23 PM, Fábio Dias fabio.d...@gmail.com wrote:
 I have optimized the GRASS vector library in trunk r64032 and added
 another topology check to v.generalize in trunk r64033. The profile of
 v.generalize now shows that it is limited by disk I/O speed (on my
 laptop with a standard laptop-like spinning HDD), which means that the
 algorithms are, under the test conditions, close to their optimum.
 This picture might change as soon as you use a high-performance server
 or a SSD.


 Then I should do a profile on my current setup.

 I have updated v.generalize again in trunk r64067. Please test the
 latest version.


 [...] the Terraclass
 shapefiles are full of errors. If you want to fix these errors, this
 will take some time.

 You know this dataset? The errors are really bugging me. It is, mostly
 due to the process/tools they usually use. We have passed over the
 request for a more topologically correct approach. Maybe on the next
 iteration. But I'll create another thread asking advice regarding
 these errors shortly :)

 I know the Terraclass dataset a bit. I used some tiles for testing. I
 was not able to import any of my test tiles without errors (after
 years of thinking about the conversion of non-topological vectors to
 topological vectors). Terraclass data are based on PRODES data, which
 I know pretty well. The PRODES classification also comes as shapfiles
 which are also full of errors, but these I managed to remove by
 carefully choosing the snapping threshold for v.in.ogr.

 By not previously dissolving and further doing v.clean tool=break the
 original data, I've reduced the processing time from more than 30h for
 1% to 24h to 11%. With the latest release, 9% in 18h.

 9% in 18h seems promising.


 However, this whole thing got me thinking about you said on an early message:

 The check_topo function can not be executed in parallel because 1)
 topology must not be modified for several boundaries in parallel, 2)
 data are written to disk, and disk IO is by nature not parallel.

 Well, disk IO, there's not much we can do about it.

 We can here and there sometimes reduce disk IO (which I did in some of
 my recent changes).

 Markus M
___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] v.generalize: does it take forever?

[Markus Neteler]

On Wed, Jan 14, 2015 at 3:54 PM, Fábio Dias fabio.d...@gmail.com wrote:
...
 What would be the best way to do that in parallel? One mapset for each
 year? Can I run multiple v.generalizes on the same input with
 different outputs?

Yes sure.

 My first thought was to run completely separated grass processes for
 each simplification, but I didn't find a way to make it search
 something different than .grass / .grass70 for the configuration
 stuff

Maybe take a look at this approach
http://grasswiki.osgeo.org/wiki/Parallel_GRASS_jobs#Grid_Engine

but even sending different v.generalize jobs to background () should
work if you have enough RAM.

markusN
___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] v.generalize: does it take forever?

[Markus Metz]

On Sat, Jan 10, 2015 at 7:23 PM, Fábio Dias fabio.d...@gmail.com wrote:
 I have optimized the GRASS vector library in trunk r64032 and added
 another topology check to v.generalize in trunk r64033. The profile of
 v.generalize now shows that it is limited by disk I/O speed (on my
 laptop with a standard laptop-like spinning HDD), which means that the
 algorithms are, under the test conditions, close to their optimum.
 This picture might change as soon as you use a high-performance server
 or a SSD.


 Then I should do a profile on my current setup.

I have updated v.generalize again in trunk r64067. Please test the
latest version.


 [...] the Terraclass
 shapefiles are full of errors. If you want to fix these errors, this
 will take some time.

 You know this dataset? The errors are really bugging me. It is, mostly
 due to the process/tools they usually use. We have passed over the
 request for a more topologically correct approach. Maybe on the next
 iteration. But I'll create another thread asking advice regarding
 these errors shortly :)

I know the Terraclass dataset a bit. I used some tiles for testing. I
was not able to import any of my test tiles without errors (after
years of thinking about the conversion of non-topological vectors to
topological vectors). Terraclass data are based on PRODES data, which
I know pretty well. The PRODES classification also comes as shapfiles
which are also full of errors, but these I managed to remove by
carefully choosing the snapping threshold for v.in.ogr.

 By not previously dissolving and further doing v.clean tool=break the
 original data, I've reduced the processing time from more than 30h for
 1% to 24h to 11%. With the latest release, 9% in 18h.

9% in 18h seems promising.


 However, this whole thing got me thinking about you said on an early message:

 The check_topo function can not be executed in parallel because 1)
 topology must not be modified for several boundaries in parallel, 2)
 data are written to disk, and disk IO is by nature not parallel.

 Well, disk IO, there's not much we can do about it.

We can here and there sometimes reduce disk IO (which I did in some of
my recent changes).

Markus M
___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] v.generalize: does it take forever?

[Fábio Dias]

 I have optimized the GRASS vector library in trunk r64032 and added
 another topology check to v.generalize in trunk r64033. The profile of
 v.generalize now shows that it is limited by disk I/O speed (on my
 laptop with a standard laptop-like spinning HDD), which means that the
 algorithms are, under the test conditions, close to their optimum.
 This picture might change as soon as you use a high-performance server
 or a SSD.


Then I should do a profile on my current setup. My grassdata dir is
not a disk, but a mounted ramdisk, which is, basically, ram, aka
really, really fast. It should be interesting.
By the way, it is really easy to do, at least on linux, and it should
really improve the performance for big datasets. Obviously, you'd need
a big machine too, but well, a big nail needs a big hammer.

cd ~
mkdir -p grassdata
sudo mount -t tmpfs -o size=512M tmpfs grassdata

In my case, the machine has 128Gb, so I made a 32Gb ramdisk. Each
vector directory has 6Gb, so it is plenty.
Of course, the data will be lost if you shutdown or reboot the
machine, so extra care is needed.
I did not compare the result with and without the ramdisk btw.


 The speed improvement is non-linear: for small datasets as in the
 official GRASS datasets, you won't notice a difference. For one tile
 of Terraclass, the processing speed should be about 2-4 times faster
 than before. For the full Terraclass dataset, the processing speed
 could be 10 times faster than before. You will need to wait until say
 10% of the processing has been reached in order to estimate the total
 processing time. Simplifying each line takes its own time, therefore
 the processing time of 100% is not equal to 100 x the processing time
 of 1%.

Indeed, but it was a (very) rough approximation.

 Another user has applied v.generalize to NLCD2011 and it took nearly 2
 months. Your dataset is probably a bit smaller, but the Terraclass
 shapefiles are full of errors. If you want to fix these errors, this
 will take some time.

You know this dataset? The errors are really bugging me. It is, mostly
due to the process/tools they usually use. We have passed over the
request for a more topologically correct approach. Maybe on the next
iteration. But I'll create another thread asking advice regarding
these errors shortly :)

 I recommend to test the new v.generalize first on a subregion of
 Terraclass. Only if the processing speed and the results are
 acceptable, proceed with the full dataset. Otherwise, please report.

Testing before deploying? Where's the fun in that ? :)
Joking aside, I did that before trying the full dataset. I did,
however interrupt the processing to start over with the new trunk
version, because you said it would be faster. And indeed it is, thank
you very much.
By not previously dissolving and further doing v.clean tool=break the
original data, I've reduced the processing time from more than 30h for
1% to 24h to 11%. With the latest release, 9% in 18h.

However, this whole thing got me thinking about you said on an early message:

 The check_topo function can not be executed in parallel because 1)
 topology must not be modified for several boundaries in parallel, 2)
 data are written to disk, and disk IO is by nature not parallel.

Well, disk IO, there's not much we can do about it. On high end
servers, again, I'm thinking big hammers, this shouldn't really be a
bottleneck nor lock the threads for long, between the disk speed and
cache, this should barely lock each thread. Assuming the vector
access functions to be thread safe (which I think they will
eventually be, IMHO it would be the first step to make the whole
software parallel-capable), we could allow parallel changes in the
topology by carefully choosing which lines are going to be considered
at a time. One simple example might be lines whose bounding boxes do
not intercept. Not sure how much overhead this would cause, or if it
would be worth it.


Thanks again,

F
___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] v.generalize: does it take forever?

[Markus Metz]

On Sun, Jan 4, 2015 at 10:45 PM, Fábio Dias fabio.d...@gmail.com wrote:
 As promised, profile of v.generalize, as of r63952.
 (The data might not be exactly the same, I might have run v.clean somewhere).

Thanks for your thorough code analysis!

My initial guess was wrong, Vect_line_intersection2() is not the
limiting factor. The R tree is also used to feed
Vect_line_intersection2(), but here it seems to be no bottleneck. The
limit was Vect_rewrite_line() and the functions called by it.

I have optimized the GRASS vector library in trunk r64032 and added
another topology check to v.generalize in trunk r64033. The profile of
v.generalize now shows that it is limited by disk I/O speed (on my
laptop with a standard laptop-like spinning HDD), which means that the
algorithms are, under the test conditions, close to their optimum.
This picture might change as soon as you use a high-performance server
or a SSD.

The speed improvement is non-linear: for small datasets as in the
official GRASS datasets, you won't notice a difference. For one tile
of Terraclass, the processing speed should be about 2-4 times faster
than before. For the full Terraclass dataset, the processing speed
could be 10 times faster than before. You will need to wait until say
10% of the processing has been reached in order to estimate the total
processing time. Simplifying each line takes its own time, therefore
the processing time of 100% is not equal to 100 x the processing time
of 1%.

Another user has applied v.generalize to NLCD2011 and it took nearly 2
months. Your dataset is probably a bit smaller, but the Terraclass
shapefiles are full of errors. If you want to fix these errors, this
will take some time.

I recommend to test the new v.generalize first on a subregion of
Terraclass. Only if the processing speed and the results are
acceptable, proceed with the full dataset. Otherwise, please report.

Markus M


 I still have the raw profiles, if anyone wants them.

 F
 -=--=-=-
 Fábio Augusto Salve Dias
 http://sites.google.com/site/fabiodias/


 On Sun, Jan 4, 2015 at 6:01 PM, Fábio Dias fabio.d...@gmail.com wrote:
 Attached is pdf generated with google-perf of v.generalize, using
 g7b4. I'm running it again for trunk.
 -=--=-=-
 Fábio Augusto Salve Dias
 http://sites.google.com/site/fabiodias/


 On Sun, Jan 4, 2015 at 5:54 PM, Markus Metz
 markus.metz.gisw...@gmail.com wrote:
 On Wed, Dec 31, 2014 at 5:20 PM, Fábio Dias fabio.d...@gmail.com wrote:

 I fussed about the v.generalize code, thinking about pthread
 parallelization. The geometry part of the code is *really* fast and
 can be easily parallelized so it can run even faster. But, according
 to the profile google-perf gave me, the real bottleneck is inside the
 check_topo function (which uses static vars and inserts a new line
 into the vector, not only checks if it breaks topo - got stuck a while
 in there due to the misnomer). More specifically in the Rtree function
 used to check if one line intersects other lines.


 The function used in check_topo is Vect_line_intersection() which does
 much more than just testing for intersections. The process could be
 made much faster if Vect_line_check_intersection() would be modified
 such that connections by end points are ignored. But I don't know if
 this would break other modules or other functionality.

 Markus M
___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] v.generalize: does it take forever?

[Fábio Dias]

Another info update (and shameless bump)
Around 18M primitives and 130M vertices.

And I'm left wondering... Has nobody tried to generalize this amount
of data yet? Or I'm going about this on the wrong way?

I even mounted the grassdata dir as a ramdisk to try to speed up the
process, but it is still way too slow...

Thanks for everything,
F
-=--=-=-
Fábio Augusto Salve Dias
http://sites.google.com/site/fabiodias/


On Wed, Jan 7, 2015 at 2:12 PM, Fábio Dias fabio.d...@gmail.com wrote:
 Another interesting update
 I believed that doing a dissolve before generalizing would speed up
 the process, because it would remove a lot of edges. The data is very
 segmented, stitching would be the right term, I suppose.

 Turns out, that belief is really wrong. The really expensive part of
 the code is checking if the new line intersect with other lines. To
 reduce the comparisons, it check the bounding boxes.
 By dissolving, I turned all the small lines into really, really big
 ones. Then all bounding boxes intercept and the algorithm does a whole
 lot more comparisons

 750 minutes of processing, 5% progress, reduction method.

 F


 -=--=-=-
 Fábio Augusto Salve Dias
 http://sites.google.com/site/fabiodias/


 On Tue, Jan 6, 2015 at 3:48 PM, Fábio Dias fabio.d...@gmail.com wrote:
 Original:
 GRASS 7.1.svn (brasil):~  v.info map=tc10
  
 ++
  | Name:tc10 
  |
  | Mapset:  terraclass   
  |
  | Location:brasil   
  |
  | Database:/home/externo/fabioasd/grass 
  |
  | Title:
  |
  | Map scale:   1:1  
  |
  | Name of creator: fabioasd 
  |
  | Organization: 
  |
  | Source date: Sat Jan  3 23:38:40 2015 
  |
  | Timestamp (first layer): none 
  |
  
 ||
  | Map format:  native   
  |
  
 ||
  |   Type of map: vector (level: 2)  
  |
  |   
  |
  |   Number of points:   0   Number of centroids:  5323741   
  |
  |   Number of lines:0   Number of boundaries: 12889264  
  |
  |   Number of areas:5573197 Number of islands:1332382   
  |
  |   
  |
  |   Map is 3D:  No  
  |
  |   Number of dblinks:  1   
  |
  |   
  |
  |   Projection: Latitude-Longitude  
  |
  |   
  |
  |   N:   5:16:18.443667NS:  18:02:29.687783S
  |
  |   E:  43:59:58.760386WW:  73:59:29.009623W
  |
  |   
  |
  |   Digitization threshold: 0   
  |
  |   Comment:
  |
  |   
  |
  
 ++


 After dissolve:

 ++
  | Name:tc10d
  |
  | Mapset:  terraclass   
  |
  | Location:brasil   
  |
  | Database:/home/externo/fabioasd/grass 
  |
  | Title:
  |
  | Map scale:   1:1  
  |
  | Name of creator: fabioasd 
  |
  | Organization: 
  |
  | Source date: Sat Jan  3 23:38:40 2015 
  |
  | Timestamp (first layer): none 
  |
  
 

Re: [GRASS-user] v.generalize: does it take forever?

[Fábio Dias]

Another interesting update
I believed that doing a dissolve before generalizing would speed up
the process, because it would remove a lot of edges. The data is very
segmented, stitching would be the right term, I suppose.

Turns out, that belief is really wrong. The really expensive part of
the code is checking if the new line intersect with other lines. To
reduce the comparisons, it check the bounding boxes.
By dissolving, I turned all the small lines into really, really big
ones. Then all bounding boxes intercept and the algorithm does a whole
lot more comparisons

750 minutes of processing, 5% progress, reduction method.

F


-=--=-=-
Fábio Augusto Salve Dias
http://sites.google.com/site/fabiodias/


On Tue, Jan 6, 2015 at 3:48 PM, Fábio Dias fabio.d...@gmail.com wrote:
 Original:
 GRASS 7.1.svn (brasil):~  v.info map=tc10
  
 ++
  | Name:tc10  
 |
  | Mapset:  terraclass
 |
  | Location:brasil
 |
  | Database:/home/externo/fabioasd/grass  
 |
  | Title: 
 |
  | Map scale:   1:1   
 |
  | Name of creator: fabioasd  
 |
  | Organization:  
 |
  | Source date: Sat Jan  3 23:38:40 2015  
 |
  | Timestamp (first layer): none  
 |
  
 ||
  | Map format:  native
 |
  
 ||
  |   Type of map: vector (level: 2)   
 |
  |
 |
  |   Number of points:   0   Number of centroids:  5323741
 |
  |   Number of lines:0   Number of boundaries: 12889264   
 |
  |   Number of areas:5573197 Number of islands:1332382
 |
  |
 |
  |   Map is 3D:  No   
 |
  |   Number of dblinks:  1
 |
  |
 |
  |   Projection: Latitude-Longitude   
 |
  |
 |
  |   N:   5:16:18.443667NS:  18:02:29.687783S 
 |
  |   E:  43:59:58.760386WW:  73:59:29.009623W 
 |
  |
 |
  |   Digitization threshold: 0
 |
  |   Comment: 
 |
  |
 |
  
 ++


 After dissolve:

 ++
  | Name:tc10d 
 |
  | Mapset:  terraclass
 |
  | Location:brasil
 |
  | Database:/home/externo/fabioasd/grass  
 |
  | Title: 
 |
  | Map scale:   1:1   
 |
  | Name of creator: fabioasd  
 |
  | Organization:  
 |
  | Source date: Sat Jan  3 23:38:40 2015  
 |
  | Timestamp (first layer): none  
 |
  
 ||
  | Map format:  native
 |
  
 ||
  |   Type of map: vector (level: 2)   
 |
  |
 |
  |   Number of points:   0   Number of centroids:  5120039
 |
  |   Number of lines:0   Number of boundaries: 

Re: [GRASS-user] v.generalize: does it take forever?

[Fábio Dias]

Original:
GRASS 7.1.svn (brasil):~  v.info map=tc10
 ++
 | Name:tc10  |
 | Mapset:  terraclass|
 | Location:brasil|
 | Database:/home/externo/fabioasd/grass  |
 | Title: |
 | Map scale:   1:1   |
 | Name of creator: fabioasd  |
 | Organization:  |
 | Source date: Sat Jan  3 23:38:40 2015  |
 | Timestamp (first layer): none  |
 ||
 | Map format:  native|
 ||
 |   Type of map: vector (level: 2)   |
 ||
 |   Number of points:   0   Number of centroids:  5323741|
 |   Number of lines:0   Number of boundaries: 12889264   |
 |   Number of areas:5573197 Number of islands:1332382|
 ||
 |   Map is 3D:  No   |
 |   Number of dblinks:  1|
 ||
 |   Projection: Latitude-Longitude   |
 ||
 |   N:   5:16:18.443667NS:  18:02:29.687783S |
 |   E:  43:59:58.760386WW:  73:59:29.009623W |
 ||
 |   Digitization threshold: 0|
 |   Comment: |
 ||
 ++


After dissolve:

++
 | Name:tc10d |
 | Mapset:  terraclass|
 | Location:brasil|
 | Database:/home/externo/fabioasd/grass  |
 | Title: |
 | Map scale:   1:1   |
 | Name of creator: fabioasd  |
 | Organization:  |
 | Source date: Sat Jan  3 23:38:40 2015  |
 | Timestamp (first layer): none  |
 ||
 | Map format:  native|
 ||
 |   Type of map: vector (level: 2)   |
 ||
 |   Number of points:   0   Number of centroids:  5120039|
 |   Number of lines:0   Number of boundaries: 12641473   |
 |   Number of areas:5369494 Number of islands:1366321|
 ||
 |   Map is 3D:  No   |
 |   Number of dblinks:  1|
 ||
 |   Projection: Latitude-Longitude   |
 ||
 |   N:   5:16:18.443667NS:  18:02:29.687783S |
 |   E:  43:59:58.760386WW:  73:59:29.009623W |
 ||
 |   Digitization threshold: 0|
 |   Comment:

Re: [GRASS-user] v.generalize: does it take forever?

[Fábio Dias]

Just for further reference, the v.dissolve takes around 24h in this
dataset. I'll post the v.info of both as soon as it is finished.

Any other ideas? I have a fairly powerful server at my disposal, but
I'm out of ideas...
-=--=-=-
Fábio Augusto Salve Dias
http://sites.google.com/site/fabiodias/


On Sun, Jan 4, 2015 at 7:45 PM, Fábio Dias fabio.d...@gmail.com wrote:
 As promised, profile of v.generalize, as of r63952.
 (The data might not be exactly the same, I might have run v.clean somewhere).

 I still have the raw profiles, if anyone wants them.

 F
 -=--=-=-
 Fábio Augusto Salve Dias
 http://sites.google.com/site/fabiodias/


 On Sun, Jan 4, 2015 at 6:01 PM, Fábio Dias fabio.d...@gmail.com wrote:
 Attached is pdf generated with google-perf of v.generalize, using
 g7b4. I'm running it again for trunk.
 -=--=-=-
 Fábio Augusto Salve Dias
 http://sites.google.com/site/fabiodias/


 On Sun, Jan 4, 2015 at 5:54 PM, Markus Metz
 markus.metz.gisw...@gmail.com wrote:
 On Wed, Dec 31, 2014 at 5:20 PM, Fábio Dias fabio.d...@gmail.com wrote:

 I fussed about the v.generalize code, thinking about pthread
 parallelization. The geometry part of the code is *really* fast and
 can be easily parallelized so it can run even faster. But, according
 to the profile google-perf gave me, the real bottleneck is inside the
 check_topo function (which uses static vars and inserts a new line
 into the vector, not only checks if it breaks topo - got stuck a while
 in there due to the misnomer). More specifically in the Rtree function
 used to check if one line intersects other lines.


 The function used in check_topo is Vect_line_intersection() which does
 much more than just testing for intersections. The process could be
 made much faster if Vect_line_check_intersection() would be modified
 such that connections by end points are ignored. But I don't know if
 this would break other modules or other functionality.

 Markus M
___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] v.generalize: does it take forever?

[Markus Metz]

On Wed, Dec 31, 2014 at 5:20 PM, Fábio Dias fabio.d...@gmail.com wrote:

 I fussed about the v.generalize code, thinking about pthread
 parallelization. The geometry part of the code is *really* fast and
 can be easily parallelized so it can run even faster. But, according
 to the profile google-perf gave me, the real bottleneck is inside the
 check_topo function (which uses static vars and inserts a new line
 into the vector, not only checks if it breaks topo - got stuck a while
 in there due to the misnomer). More specifically in the Rtree function
 used to check if one line intersects other lines.


The function used in check_topo is Vect_line_intersection() which does
much more than just testing for intersections. The process could be
made much faster if Vect_line_check_intersection() would be modified
such that connections by end points are ignored. But I don't know if
this would break other modules or other functionality.

Markus M
___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] v.generalize: does it take forever?

[Fábio Dias]

Attached is pdf generated with google-perf of v.generalize, using
g7b4. I'm running it again for trunk.
-=--=-=-
Fábio Augusto Salve Dias
http://sites.google.com/site/fabiodias/


On Sun, Jan 4, 2015 at 5:54 PM, Markus Metz
markus.metz.gisw...@gmail.com wrote:
 On Wed, Dec 31, 2014 at 5:20 PM, Fábio Dias fabio.d...@gmail.com wrote:

 I fussed about the v.generalize code, thinking about pthread
 parallelization. The geometry part of the code is *really* fast and
 can be easily parallelized so it can run even faster. But, according
 to the profile google-perf gave me, the real bottleneck is inside the
 check_topo function (which uses static vars and inserts a new line
 into the vector, not only checks if it breaks topo - got stuck a while
 in there due to the misnomer). More specifically in the Rtree function
 used to check if one line intersects other lines.


 The function used in check_topo is Vect_line_intersection() which does
 much more than just testing for intersections. The process could be
 made much faster if Vect_line_check_intersection() would be modified
 such that connections by end points are ignored. But I don't know if
 this would break other modules or other functionality.

 Markus M


v.gen.profile.pdf
Description: Adobe PDF document
___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] v.generalize: does it take forever?

[Fábio Dias]

As promised, profile of v.generalize, as of r63952.
(The data might not be exactly the same, I might have run v.clean somewhere).

I still have the raw profiles, if anyone wants them.

F
-=--=-=-
Fábio Augusto Salve Dias
http://sites.google.com/site/fabiodias/


On Sun, Jan 4, 2015 at 6:01 PM, Fábio Dias fabio.d...@gmail.com wrote:
 Attached is pdf generated with google-perf of v.generalize, using
 g7b4. I'm running it again for trunk.
 -=--=-=-
 Fábio Augusto Salve Dias
 http://sites.google.com/site/fabiodias/


 On Sun, Jan 4, 2015 at 5:54 PM, Markus Metz
 markus.metz.gisw...@gmail.com wrote:
 On Wed, Dec 31, 2014 at 5:20 PM, Fábio Dias fabio.d...@gmail.com wrote:

 I fussed about the v.generalize code, thinking about pthread
 parallelization. The geometry part of the code is *really* fast and
 can be easily parallelized so it can run even faster. But, according
 to the profile google-perf gave me, the real bottleneck is inside the
 check_topo function (which uses static vars and inserts a new line
 into the vector, not only checks if it breaks topo - got stuck a while
 in there due to the misnomer). More specifically in the Rtree function
 used to check if one line intersects other lines.


 The function used in check_topo is Vect_line_intersection() which does
 much more than just testing for intersections. The process could be
 made much faster if Vect_line_check_intersection() would be modified
 such that connections by end points are ignored. But I don't know if
 this would break other modules or other functionality.

 Markus M


tr.pdf
Description: Adobe PDF document
___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] v.generalize: does it take forever?

[Markus Metz]

On Wed, Dec 31, 2014 at 5:20 PM, Fábio Dias fabio.d...@gmail.com wrote:
 On Wed, Dec 31, 2014 at 12:23 PM, Markus Neteler nete...@osgeo.org wrote:
 On Sun, Dec 28, 2014 at 8:04 PM, Fábio Dias fabio.d...@gmail.com wrote:
 Hello all,

 Context: I've loaded some shp files into postgis, containing
 information over the amazon forest. For reference, the sql script has
 around 6Gb.

 How many polygons do you have there approximately?

 That depends. The information is separated by states. In this case, AP
 corresponds to the state of Amapá, which is the smallest one,
 datawise, with only 70ish mb. The state of Pará has 2.3gb.
 Ideally, I would generalize the information as a whole, not each state
 independently, so I don't get gaps/etc.

Makes sense.

 The whole thing, for one of
 the 4 years available, has around 5M polygons (counting from postgis,
 I do not have the data imported on grass at the moment. I'm importing,
 but it will take a while). The other years have more polygons, and it
 wouldn't be unreasonable to expect around 10M.

I would avoid the postgis step and import the shapefiles directly to GRASS.


 Problem: I managed do import, clean and dissolve properly, but when I
 run the generalization, by my estimates, it would take almost an year
 to complete.

 This will also depend on the generalization method you selected.

 Yes, but in a minor way, as I'll detail in the next part.


 I also noticed that neither grass nor postgis are capable of parallel
 processing...
 Yeah, hot topic for 2015 :-) Indeed, worth a thesis in my view!

 I fussed about the v.generalize code, thinking about pthread
 parallelization. The geometry part of the code is *really* fast and
 can be easily parallelized so it can run even faster. But, according
 to the profile google-perf gave me, the real bottleneck is inside the
 check_topo function (which uses static vars and inserts a new line
 into the vector, not only checks if it breaks topo - got stuck a while
 in there due to the misnomer). More specifically in the Rtree function
 used to check if one line intersects other lines.

The check_topo function can not be executed in parallel because 1)
topology must not be modified for several boundaries in parallel, 2)
data are written to disk, and disk IO is by nature not parallel.


 I commented out the check_topo call and it ran a whole lot faster. The
 result, obviously, was really bad and messed up, topologically, but it
 confirmed that it is indeed the bottleneck.

 Question: Am I using the correct tool for that? Is there a way to
 speed up the processing?

 For reference, the commands i've used (grass70, beta4, 22 dez 2014):

 (Glad you use beta4, so we have a recent code base to check.)

A pity you use beta4, please use current trunk, because there are a
few improvements in trunk not available in beta4. v.generalize should
be quite a bit faster in trunk than in beta4.


 I suppose that method=douglas is faster that method=reduction?

Yes.


 With the full dataset, both were painfully slow. And by slow, I mean
 more than 24h without printing the 1% message slow.


 What is the projection you are working with? Given the threshold and
 assuming LatLong, I get a short distance:

 4674. It is indeed latlong.

That does not matter.

 The idea is to have multiple generalizations as different tables on
 postgis and fetch data from the correct table using the current zoom
 level in the web interface (googlemaps based). I considered serving
 the map using wms/geoserver and also rendering on the client using
 node.js (io.js now, apparently) and topojson.

As you mentioned above, the whole dataset should be generalized at
once to avoid gaps and overlapping parts.


 Probably you want to try a larger threshold first?

No, rather try a very small threshold first.

Markus M
___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] v.generalize: does it take forever?

[Fábio Dias]

On Wed, Dec 31, 2014 at 12:23 PM, Markus Neteler nete...@osgeo.org wrote:
 On Sun, Dec 28, 2014 at 8:04 PM, Fábio Dias fabio.d...@gmail.com wrote:
 Hello all,

 Context: I've loaded some shp files into postgis, containing
 information over the amazon forest. For reference, the sql script has
 around 6Gb.

 How many polygons do you have there approximately?

That depends. The information is separated by states. In this case, AP
corresponds to the state of Amapá, which is the smallest one,
datawise, with only 70ish mb. The state of Pará has 2.3gb.
Ideally, I would generalize the information as a whole, not each state
independently, so I don't get gaps/etc. The whole thing, for one of
the 4 years available, has around 5M polygons (counting from postgis,
I do not have the data imported on grass at the moment. I'm importing,
but it will take a while). The other years have more polygons, and it
wouldn't be unreasonable to expect around 10M.


 Problem: I managed do import, clean and dissolve properly, but when I
 run the generalization, by my estimates, it would take almost an year
 to complete.

 This will also depend on the generalization method you selected.

Yes, but in a minor way, as I'll detail in the next part.


 I also noticed that neither grass nor postgis are capable of parallel
 processing...
 Yeah, hot topic for 2015 :-) Indeed, worth a thesis in my view!

I fussed about the v.generalize code, thinking about pthread
parallelization. The geometry part of the code is *really* fast and
can be easily parallelized so it can run even faster. But, according
to the profile google-perf gave me, the real bottleneck is inside the
check_topo function (which uses static vars and inserts a new line
into the vector, not only checks if it breaks topo - got stuck a while
in there due to the misnomer). More specifically in the Rtree function
used to check if one line intersects other lines.

I commented out the check_topo call and it ran a whole lot faster. The
result, obviously, was really bad and messed up, topologically, but it
confirmed that it is indeed the bottleneck.

 Question: Am I using the correct tool for that? Is there a way to
 speed up the processing?

 For reference, the commands i've used (grass70, beta4, 22 dez 2014):

 (Glad you use beta4, so we have a recent code base to check.)

 v.in.ogr -e --verbose input=pg:host=localhost (...) layer=ap10
 output=ap10 snap=1e-6
 -- Please tell us how many polygons or lines the layer ap10 contains.

ap10 was just a 'toy' dataset to try out the script. It is
considerably smaller than the real dataset. The postgis table of this
data has 50k records/polygons.
v.info for it on: http://pastebin.com/8RZELd8p


 v.clean -c --verbose input=ap10 output=ap10c tool=bpol,break,rmsa type=line
 -- Not sure but should type be boundary or line?

I tried combinations and variations, I'm not that sure either. My
postgis data is composed of polygons. It is a landuse classification
data (or something like this, I'm not that familiar with
geo-nomenclature).

 v.dissolve --verbose input=ap10c column=tc_2010 output=ap10d --overwrite
 -- How many polygons does ap10d contain?

120k boundaries (http://pastebin.com/8RZELd8p)


 Try #1 )   v.generalize --verbose --overwrite input=ap10d output=ap10r
 method=reduction threshold=0.00025 --overwrite
 Try #2 )   v.generalize --verbose --overwrite input=ap10d output=ap10g
 method=douglas threshold=0.00025 --overwrite

 I suppose that method=douglas is faster that method=reduction?

With the full dataset, both were painfully slow. And by slow, I mean
more than 24h without printing the 1% message slow.


 What is the projection you are working with? Given the threshold and
 assuming LatLong, I get a short distance:

4674. It is indeed latlong.
The idea is to have multiple generalizations as different tables on
postgis and fetch data from the correct table using the current zoom
level in the web interface (googlemaps based). I considered serving
the map using wms/geoserver and also rendering on the client using
node.js (io.js now, apparently) and topojson.


 GRASS 7.1.svn (latlong):~  g.region -g res=0.00025 -a
 n=4.15
 s=-16.369
 w=-76.23975
 e=-45.1125
 nsres=0.00025
 ewres=0.00025
 ...

 GRASS 7.1.svn (latlong):~  g.region -m...
 nsres=27.64959657
 ewres=27.21883498
 ...

 Probably you want to try a larger threshold first?

Empirically, that valued removed only the jagged edges, so it was a
good first generalization. My idea was that, afterwards, I'd increase
the threshold and generate more generalizations.

thanks again,
F
___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] v.generalize dateline wrap problem

[Jed O . Kaplan]

Dear all,

Further to my question, here is the output of g.version:

GRASS 7.1.svn (global_lonlat):~  g.version -g
version=7.1.svn
date=2014
revision=62762
build_date=2014-11-17
build_platform=x86_64-apple-darwin10.8.0

Thanks,

Jed

On 17 Nov 2014, at 15:34, Markus Neteler nete...@osgeo.org wrote:

 Hi,
 
 can you please send also the output of
 
 g.version -g
 
 to the list? 
 ​
 thanks
 Markus

___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] v.generalize dateline wrap problem

[Markus Neteler]

On Mon, Nov 17, 2014 at 3:37 PM, Jed O. Kaplan jed.kap...@unil.ch wrote:
 Dear all,

 Further to my question, here is the output of g.version:

 GRASS 7.1.svn (global_lonlat):~  g.version -g
 version=7.1.svn
 date=2014
 revision=62762
 build_date=2014-11-17

I can confirm the issue also in G7.0.svn on Linux.

Markus
___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] v.generalize threshold and boyle's algorithm

[Johannes Radinger]

Hi Markus,

 Original-Nachricht 
 Datum: Tue, 7 Feb 2012 18:00:26 +0100
 Von: Markus Metz markus.metz.gisw...@googlemail.com
 An: Johannes Radinger jradin...@gmx.at
 CC: grASS user list grass-user@lists.osgeo.org
 Betreff: Re: [GRASS-user] v.generalize threshold and boyle\'s algorithm

 Johannes Radinger wrote:
  Hello,
 
  after some month a tried an script again which
  calls v.generalize. It seems that this module
  has changed in version 6.4.2 and 6.5SVN.
 
  Before I could use following:
  grass.run_command(v.generalize,
                       input = input,
                       output = vector_gen,
                       method = boyle,
                       look_ahead = 8,
                       overwrite = True)
 
  Now it seems that v.generalize needs a threshold value (required!)
  although the manual states only: Input parameters: input, look_ahead
 
 Right, the module requires threshold, but Boyle (and some other
 methods) do not require threshold, instead they require other
 parameters. Threshold being required is maybe annoying, but not that
 serious, more serious is that the module does not check if all
 parameters for the selected method are specified.
 
  I am confused now if this value is really necessary for calculating
 boyle's algorithm and if yes how should a value be choosen?
 
 Threshold is not used by boyle, just give the module some dummy value.

Thank you for that answer. Of course its not of big deal to provide a dummy 
value, I was just confused. 

/johannes

 
  Moreover I'd like to know what means the 'r' flag (backwards
 compatibility), were there any major changes in recent time (last year)?
 
 A number of bugs were fixed because v.generalize produced corrupt
 output in 6.4.1 (see e.g. recent ml thread [1]).
 
 Markus M
 
 [1] http://lists.osgeo.org/pipermail/grass-user/2012-January/063327.html

-- 
NEU: FreePhone 3-fach-Flat mit kostenlosem Smartphone!  

Jetzt informieren: http://www.gmx.net/de/go/freephone/
___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] v.generalize threshold and boyle's algorithm

[Markus Metz]

Johannes Radinger wrote:
 Hello,

 after some month a tried an script again which
 calls v.generalize. It seems that this module
 has changed in version 6.4.2 and 6.5SVN.

 Before I could use following:
 grass.run_command(v.generalize,
                      input = input,
                      output = vector_gen,
                      method = boyle,
                      look_ahead = 8,
                      overwrite = True)

 Now it seems that v.generalize needs a threshold value (required!)
 although the manual states only: Input parameters: input, look_ahead

Right, the module requires threshold, but Boyle (and some other
methods) do not require threshold, instead they require other
parameters. Threshold being required is maybe annoying, but not that
serious, more serious is that the module does not check if all
parameters for the selected method are specified.

 I am confused now if this value is really necessary for calculating boyle's 
 algorithm and if yes how should a value be choosen?

Threshold is not used by boyle, just give the module some dummy value.

 Moreover I'd like to know what means the 'r' flag (backwards compatibility), 
 were there any major changes in recent time (last year)?

A number of bugs were fixed because v.generalize produced corrupt
output in 6.4.1 (see e.g. recent ml thread [1]).

Markus M

[1] http://lists.osgeo.org/pipermail/grass-user/2012-January/063327.html
___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] v.generalize: strange behaviour on win?

[Markus Metz]

On Sun, Jan 29, 2012 at 1:12 PM, Paolo Cavallini cavall...@faunalia.it wrote:
 Il 28/01/2012 11:58, Markus Metz ha scritto:

 are left have mixed up attributes. That was fixed such that boundaries
 are not generalized if the generalization would damage topology. For


 OK, now I see. I think adding a warning in these case should be useful. Is 
 this
 documented in the man?

These cases are reported in verbose mode. This is not mentioned in the
manual. BTW, v.generalize in 6.4.2 and above behaves like v.clean
tool=prune since version 5.x, but offers far more methods. The
topology checks and skipping some boundaries if it is not possible to
safely generalize them is in this sense not new, but has been there
for many years.

 However, in my case topology is maintained, even if the areas are obviously 
 overly
 simplistic.

My previous description was obviously incomplete. Output topology is
always correct, independent of the version. In 6.4.1, topologically
incorrect boundaries are deleted at the end of the module. In 6.4.2
and above, generalized boundaries are tested if they would damage
topology, if no, fine, if yes, the original is kept. Because in 6.4.1
the original boundaries are not kept and all incorrect generalized
boundaries are deleted, the output may suffer from heavy losses, i.e.
areas disappear (see nc example mentioned earlier and also your own
example).

Markus M

On the other hand, I confirm that the attributes are mixed up.
 Thanks a lot for the explanation.
 All the best.
 --
 Paolo Cavallini - Faunalia
 www.faunalia.eu
 Full contact details at www.faunalia.eu/pc
___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] v.generalize: strange behaviour on win?

[Paolo Cavallini]

Il 30/01/2012 16:35, Markus Metz ha scritto:

 My previous description was obviously incomplete. Output topology is
 always correct, independent of the version. In 6.4.1, topologically

Thanks Markus, this clarifies the issue a lot.
All the best.
-- 
Paolo Cavallini - Faunalia
www.faunalia.eu
Full contact details at www.faunalia.eu/pc
___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] v.generalize: strange behaviour on win?

[Markus Neteler]

On Mon, Jan 30, 2012 at 4:56 PM, Paolo Cavallini cavall...@faunalia.it wrote:
 Il 30/01/2012 16:35, Markus Metz ha scritto:

 My previous description was obviously incomplete. Output topology is
 always correct, independent of the version. In 6.4.1, topologically

 Thanks Markus, this clarifies the issue a lot.

Please add important notes also to
http://grass.osgeo.org/wiki/V.generalize_tutorial

thanks
MarkusN
___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] v.generalize: strange behaviour on win?

[Paolo Cavallini]

Il 28/01/2012 11:36, Martin Landa ha scritto:

 type=area ... type=boundary

 ...no good!

 
 AFAIK the parser reads this combination as `type=area,boundary` in this case.

the results are the same when using ara+boundary or boundary alone.
thanks.
-- 
Paolo Cavallini - Faunalia
www.faunalia.eu
Full contact details at www.faunalia.eu/pc
___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] v.generalize: strange behaviour on win?

[Markus Metz]

On Sat, Jan 28, 2012 at 9:12 AM, Paolo Cavallini cavall...@faunalia.it wrote:
 Il 27/01/2012 19:24, Markus Metz ha scritto:

 v.generalize is broken in 6.4.1

 oh, so it's the different version - I didn't know v.generalize was changed 
 recently,
 thanks for letting me know. strangely enough, in Linux (6.4.1) it works well, 
 whereas
 in Windows (AFAIK 6.4.2RC3) it does not.
 Any explanation?

Yes. The quality of the result is judged by the integrity of the
output data. Therefore the result of 6.4.1 is bad and the result of
6.4.2 is good, contrary to your interpretation. You can try 6.4.1 with
the boundary_county vector in the North Carolina dataset and a
threshold of 100. Half of North Carolina disappears, the areas that
are left have mixed up attributes. That was fixed such that boundaries
are not generalized if the generalization would damage topology. For
your example vector I would recommend a much smaller threshold,
starting with 10, then maybe 20. You could easily end up with a much
higher reduction that with threshold = 10.

Markus M
___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] v.generalize: strange behaviour on win?

[Paolo Cavallini]

Il 26/01/2012 20:53, Markus Metz ha scritto:

 You would need to provide the input, not the output as an example.

https://int.faunalia.it/~paolo/province.zip

 Considering the detail of the output and the threshold used, it is no
 surprise that not all boundaries were generalized because that would
 break topology. If you want to generalize with a threshold of 10,
 I would recommend to first remove all areas with a size up to 10.

This is not the point. The very same input and command work in Linux, leave
ungeneralized boundaries in Windows.
If you consider that also r.to.vect for lines also behaves differently between 
the
two OSs, I suspect an OS-specific bug here.

Any confirmation?
Thanks a lot.
-- 
Paolo Cavallini - Faunalia
www.faunalia.eu
Full contact details at www.faunalia.eu/pc
___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] v.generalize: strange behaviour on win?

[Markus Metz]

On Fri, Jan 27, 2012 at 1:32 PM, Paolo Cavallini cavall...@faunalia.it wrote:
 Il 26/01/2012 20:53, Markus Metz ha scritto:

 You would need to provide the input, not the output as an example.

 https://int.faunalia.it/~paolo/province.zip

 Considering the detail of the output and the threshold used, it is no
 surprise that not all boundaries were generalized because that would
 break topology. If you want to generalize with a threshold of 10,
 I would recommend to first remove all areas with a size up to 10.

 This is not the point. The very same input and command work in Linux, leave
 ungeneralized boundaries in Windows.

I tried v.generalize on the Province_general3 shapefile on Linux with
type=boundary method=douglas threshold=10 and it did not modify
any boundaries. I also tried the input province shapefile and get the
same output. That is, if Province_general3 has been generated on
windows, I do get the same result on Linux, no difference between the
OS's. Are you using an older grass version on Linux?

Markus M
___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] v.generalize: strange behaviour on win?

[Paolo Cavallini]

Il 27/01/2012 19:02, Markus Metz ha scritto:

 same output. That is, if Province_general3 has been generated on
 windows, I do get the same result on Linux, no difference between the
 OS's. Are you using an older grass version on Linux?

aptitude show grass
Versione: 6.4.1-1
Thanks.

-- 
Paolo Cavallini - Faunalia
www.faunalia.eu
Full contact details at www.faunalia.eu/pc
___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] v.generalize: strange behaviour on win?

[Markus Metz]

On Fri, Jan 27, 2012 at 7:01 PM, Paolo Cavallini cavall...@faunalia.it wrote:
 Il 27/01/2012 19:02, Markus Metz ha scritto:

 same output. That is, if Province_general3 has been generated on
 windows, I do get the same result on Linux, no difference between the
 OS's. Are you using an older grass version on Linux?

 aptitude show grass
 Versione: 6.4.1-1

v.generalize is broken in 6.4.1

Markus M
___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] v.generalize: strange behaviour on win?

[Markus Metz]

On Wed, Jan 25, 2012 at 11:03 AM, Paolo Cavallini cavall...@faunalia.it wrote:
 Hi all.
 Commands like:

 v.generalize input=province@Mapset_stefano type=area layer=1 -c
 type=boundary method=douglas threshold=10 look_ahead=7
 reduction=50 slide=0.5 angle_thresh=3 degree_thresh=0
 closeness_thresh=0 betweeness_thresh=0 alpha=1.0 beta=1.0 iterations=1
 output=Province_general3

 on Linux go well, whereas on windows some of the features are not simplified: 
 see
 https://int.faunalia.it/~paolo/generalize.zip
 for an example.
 Anyone confirms? Should I open a bug?

You would need to provide the input, not the output as an example.
Considering the detail of the output and the threshold used, it is no
surprise that not all boundaries were generalized because that would
break topology. If you want to generalize with a threshold of 10,
I would recommend to first remove all areas with a size up to 10.

Markus M
___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] v.generalize issue

[Roger André]

Hi Markus,

That's good to know.  Can you tell me the bug number?  I'd like to review
the details of it.

Thanks,

Roger
--

On Wed, Oct 26, 2011 at 12:23 AM, Markus Metz 
markus.metz.gisw...@googlemail.com wrote:

 On Tue, Oct 25, 2011 at 11:07 PM, Roger André ran...@gmail.com wrote:
  Hello GRASS Community,
  I'm having a problem that I'm hoping you can help me find the source of.
  I think that either A) I am trying to use the v.generalize tool
 incorrectly,
  B) that my data has some sort of topological problem which I'm unaware
 of,
  or C) that there is a bug in v.generalize.  The simplest way to describe
  what's happening is that I am consistently unable to generalize a large
  group of contiguous polygons (African states most recently) without
 losing a
  large number of polygons from my data.  I have tried a variety of
 different
  things, all without success, and now I'd like to ask if someone would be
  willing to verify that what I'm seeing isn't a problem of my own
 creation.
   I've posted my data files at www.macrogis.net/africa, along with a
  before-and-after image of the specific problem I've encountered.  I'm
 using
  GRASS v 6.4.1 which I downloaded from the main website 2 months ago.

 This bug has been fixed in GRASS 6.4.2

 Markus M

___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] v.generalize issue

[Markus Metz]

Roger André wrote:
 Hi Markus,
 That's good to know.  Can you tell me the bug number?  I'd like to review
 the details of it.

There is no bug number, at least I could not find a corresponding bug
number. There were a few complaints about v.generalize in the mailing
lists. In short, you are not the first one to discover this bug.
Additionally, v.generalize was randomly swapping area attributes. This
is also fixed.

Since you are asking about a bug number, are you interested in the
effects or the causes?

Markus M


 Thanks,
 Roger
 --

 On Wed, Oct 26, 2011 at 12:23 AM, Markus Metz
 markus.metz.gisw...@googlemail.com wrote:

 On Tue, Oct 25, 2011 at 11:07 PM, Roger André ran...@gmail.com wrote:
  Hello GRASS Community,
  I'm having a problem that I'm hoping you can help me find the source of.
  I think that either A) I am trying to use the v.generalize tool
  incorrectly,
  B) that my data has some sort of topological problem which I'm unaware
  of,
  or C) that there is a bug in v.generalize.  The simplest way to describe
  what's happening is that I am consistently unable to generalize a large
  group of contiguous polygons (African states most recently) without
  losing a
  large number of polygons from my data.  I have tried a variety of
  different
  things, all without success, and now I'd like to ask if someone would be
  willing to verify that what I'm seeing isn't a problem of my own
  creation.
   I've posted my data files at www.macrogis.net/africa, along with a
  before-and-after image of the specific problem I've encountered.  I'm
  using
  GRASS v 6.4.1 which I downloaded from the main website 2 months ago.

 This bug has been fixed in GRASS 6.4.2

 Markus M


___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] v.generalize issue

[Roger André]

Ah, ok.  The causes, please.

It's a really powerful tool, and I would like to write some additional
documentation on how to use it for generalizing and smoothing large polygon
sets, but first I need to make sure that I can anticipate it's behavior a
little better.  The area attribute swapping was something that I had noticed
as well, but I figured that was just something to be expected.

Also, would it be helpful to have additional testers for some of these?  I'm
not using the raster tools at the moment, but am using the vector tools
quite a bit (console only though).

Roger
--

On Wed, Oct 26, 2011 at 11:51 AM, Markus Metz 
markus.metz.gisw...@googlemail.com wrote:

 Roger André wrote:
  Hi Markus,
  That's good to know.  Can you tell me the bug number?  I'd like to review
  the details of it.

 There is no bug number, at least I could not find a corresponding bug
 number. There were a few complaints about v.generalize in the mailing
 lists. In short, you are not the first one to discover this bug.
 Additionally, v.generalize was randomly swapping area attributes. This
 is also fixed.

 Since you are asking about a bug number, are you interested in the
 effects or the causes?

 Markus M


  Thanks,
  Roger
  --
 
  On Wed, Oct 26, 2011 at 12:23 AM, Markus Metz
  markus.metz.gisw...@googlemail.com wrote:
 
  On Tue, Oct 25, 2011 at 11:07 PM, Roger André ran...@gmail.com wrote:
   Hello GRASS Community,
   I'm having a problem that I'm hoping you can help me find the source
 of.
   I think that either A) I am trying to use the v.generalize tool
   incorrectly,
   B) that my data has some sort of topological problem which I'm unaware
   of,
   or C) that there is a bug in v.generalize.  The simplest way to
 describe
   what's happening is that I am consistently unable to generalize a
 large
   group of contiguous polygons (African states most recently) without
   losing a
   large number of polygons from my data.  I have tried a variety of
   different
   things, all without success, and now I'd like to ask if someone would
 be
   willing to verify that what I'm seeing isn't a problem of my own
   creation.
I've posted my data files at www.macrogis.net/africa, along with a
   before-and-after image of the specific problem I've encountered.  I'm
   using
   GRASS v 6.4.1 which I downloaded from the main website 2 months ago.
 
  This bug has been fixed in GRASS 6.4.2
 
  Markus M
 
 

___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] v.generalize issue

[Markus Metz]

Roger André wrote:
 Ah, ok.  The causes, please.
 It's a really powerful tool, and I would like to write some additional
 documentation on how to use it for generalizing and smoothing large polygon
 sets, but first I need to make sure that I can anticipate it's behavior a
 little better.  The area attribute swapping was something that I had noticed
 as well, but I figured that was just something to be expected.
 Also, would it be helpful to have additional testers for some of these?  I'm
 not using the raster tools at the moment, but am using the vector tools
 quite a bit (console only though).

Please use GRASS 6.4.2 for testing. A number of bugs present in 6.4.1
have been fixed in 6.4.2.

With regard to v.generalize, randomly swapping of area attributes was
a serious bug, this was by no means to be expected. This swapping
occurred because the original author used area ids instead of
categories to assign attributes to the smoothed vector.  This is
within the logic of the GRASS vector model nonsense because area ids
are randomly assigned, area categories are instead fixed because
user-defined. The disappearance of polygons happened because smoothed
boundaries that violated vector topology were deleted, after that all
boundaries that were topologically incorrect. Boundaries are now only
smoothed/simplified if the modification does not violate vector
topology.

HTH,

Markus M


 Roger
 --
 On Wed, Oct 26, 2011 at 11:51 AM, Markus Metz
 markus.metz.gisw...@googlemail.com wrote:

 Roger André wrote:
  Hi Markus,
  That's good to know.  Can you tell me the bug number?  I'd like to
  review
  the details of it.

 There is no bug number, at least I could not find a corresponding bug
 number. There were a few complaints about v.generalize in the mailing
 lists. In short, you are not the first one to discover this bug.
 Additionally, v.generalize was randomly swapping area attributes. This
 is also fixed.

 Since you are asking about a bug number, are you interested in the
 effects or the causes?

 Markus M


  Thanks,
  Roger
  --
 
  On Wed, Oct 26, 2011 at 12:23 AM, Markus Metz
  markus.metz.gisw...@googlemail.com wrote:
 
  On Tue, Oct 25, 2011 at 11:07 PM, Roger André ran...@gmail.com wrote:
   Hello GRASS Community,
   I'm having a problem that I'm hoping you can help me find the source
   of.
   I think that either A) I am trying to use the v.generalize tool
   incorrectly,
   B) that my data has some sort of topological problem which I'm
   unaware
   of,
   or C) that there is a bug in v.generalize.  The simplest way to
   describe
   what's happening is that I am consistently unable to generalize a
   large
   group of contiguous polygons (African states most recently) without
   losing a
   large number of polygons from my data.  I have tried a variety of
   different
   things, all without success, and now I'd like to ask if someone would
   be
   willing to verify that what I'm seeing isn't a problem of my own
   creation.
    I've posted my data files at www.macrogis.net/africa, along with a
   before-and-after image of the specific problem I've encountered.  I'm
   using
   GRASS v 6.4.1 which I downloaded from the main website 2 months ago.
 
  This bug has been fixed in GRASS 6.4.2
 
  Markus M
 
 


___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] v.generalize issues

[Markus Metz]

On Thu, Jun 30, 2011 at 7:49 PM, Christian Guirreri guirr...@gmail.com wrote:
 I suppose my other question is if anyone has any recommendations concerning
 v.generalize as it applies specifically to tiger data - I'm doing all of US.
 Lots of settings and tests and its a bit overwhelming for a newbie.

For simplification (vertex reduction), Douglas-Peucker is generally a
good choice, it's relatively simple and commonly used.

For smoothing, Chaiken's Algorithm and Hermite Interpolation should
produce quite different results, apart from being a matter of taste,
it depends on what kind of modifications are allowed.

Markus M


 On Jun 30, 2011 11:33 AM, Christian Guirreri christ...@guirreri.com
 wrote:
 Success! Thanks for making this easy for the total noob that I am.

 On Thu, Jun 30, 2011 at 11:04 AM, Markus Metz 
 markus.metz.gisw...@googlemail.com wrote:

 On Thu, Jun 30, 2011 at 4:47 PM, Christian Guirreri
 christ...@guirreri.com wrote:
  I'm on GRASS 6.4.1. QGIS 1.7.0
 
 In this version, v.generalize is still broken. You would need a recent
 version of GRASS 6.4.2. For Windows, you can get a recent version
 here:

 http://wingrass.fsv.cvut.cz/grass64/

 Markus M


  On Thu, Jun 30, 2011 at 10:18 AM, Markus Metz
  markus.metz.gisw...@googlemail.com wrote:
 
  On Thu, Jun 30, 2011 at 4:01 PM, Christian Guirreri
  christ...@guirreri.com wrote:
   I'm a brand new user to GIS - my goal right now is to heavily
   simplify
   Tiger
   2010 county and distrct data, without producing gaps between
 boundaries.
   I've been testing this with v.generalize in grass via QuantumGIS -
 I've
   had
   tons of issues with the Grass toolbox crashing, but have narrowed
   down
   to
   using the Hermite algorithm. While it works, I'm having some bizarre
   issues
   with it. Apologies for the cross-post with the PostGIS mailing list.
  
   In the attached gif of California counties, from left to right, I
   have
   used
   the following tolerance values with the Hermite algorithm:
   - original
   - 1.0
   - 0.08
   - 0.01
   - 0.1
  
   Why do counties disappear entirely as I decrease the tolerance?
  
  This problem has been fixed in GRASS 6.4 only 2 weeks ago (June 13).
  Please update your GRASS version if possible.
 
  Markus M
 
 
   In the Grass Tools I choose the v.generalize function. I choose
 Boundary
   as
   the feature type (though I've tried checking others, as well as all
   of
   them
   and it doesn't seem to change anything). Everything else is default,
   except
   for tolerance as notated above.
  
   When I tested this originally on only Arkansas and Mississippi, I
   got
   really
   nice results. I then tried it on the entire US and had the missing
   counties
   problem. So I tried only California, and still have the same issue.
  
   I've tried other algorithms, but this has so far given me the detail
   I
   want
   - of course sans counties! Any thoughts?
  
   Thanks,
   - Chris
   ___
   grass-user mailing list
   grass-user@lists.osgeo.org
   http://lists.osgeo.org/mailman/listinfo/grass-user
  
  
 
 


___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] v.generalize issues

[Markus Metz]

On Thu, Jun 30, 2011 at 4:01 PM, Christian Guirreri
christ...@guirreri.com wrote:
 I'm a brand new user to GIS - my goal right now is to heavily simplify Tiger
 2010 county and distrct data, without producing gaps between boundaries.
 I've been testing this with v.generalize in grass via QuantumGIS - I've had
 tons of issues with the Grass toolbox crashing, but have narrowed down to
 using the Hermite algorithm. While it works, I'm having some bizarre issues
 with it. Apologies for the cross-post with the PostGIS mailing list.

 In the attached gif of California counties, from left to right, I have used
 the following tolerance values with the Hermite algorithm:
  - original
  - 1.0
  - 0.08
  - 0.01
  - 0.1

 Why do counties disappear entirely as I decrease the tolerance?

This problem has been fixed in GRASS 6.4 only 2 weeks ago (June 13).
Please update your GRASS version if possible.

Markus M


 In the Grass Tools I choose the v.generalize function. I choose Boundary as
 the feature type (though I've tried checking others, as well as all of them
 and it doesn't seem to change anything). Everything else is default, except
 for tolerance as notated above.

 When I tested this originally on only Arkansas and Mississippi, I got really
 nice results. I then tried it on the entire US and had the missing counties
 problem. So I tried only California, and still have the same issue.

 I've tried other algorithms, but this has so far given me the detail I want
 - of course sans counties! Any thoughts?

 Thanks,
  - Chris
 ___
 grass-user mailing list
 grass-user@lists.osgeo.org
 http://lists.osgeo.org/mailman/listinfo/grass-user


___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] v.generalize issues

[Christian Guirreri]

I'm on GRASS 6.4.1. QGIS 1.7.0

On Thu, Jun 30, 2011 at 10:18 AM, Markus Metz 
markus.metz.gisw...@googlemail.com wrote:

 On Thu, Jun 30, 2011 at 4:01 PM, Christian Guirreri
 christ...@guirreri.com wrote:
  I'm a brand new user to GIS - my goal right now is to heavily simplify
 Tiger
  2010 county and distrct data, without producing gaps between boundaries.
  I've been testing this with v.generalize in grass via QuantumGIS - I've
 had
  tons of issues with the Grass toolbox crashing, but have narrowed down to
  using the Hermite algorithm. While it works, I'm having some bizarre
 issues
  with it. Apologies for the cross-post with the PostGIS mailing list.
 
  In the attached gif of California counties, from left to right, I have
 used
  the following tolerance values with the Hermite algorithm:
   - original
   - 1.0
   - 0.08
   - 0.01
   - 0.1
 
  Why do counties disappear entirely as I decrease the tolerance?
 
 This problem has been fixed in GRASS 6.4 only 2 weeks ago (June 13).
 Please update your GRASS version if possible.

 Markus M


  In the Grass Tools I choose the v.generalize function. I choose Boundary
 as
  the feature type (though I've tried checking others, as well as all of
 them
  and it doesn't seem to change anything). Everything else is default,
 except
  for tolerance as notated above.
 
  When I tested this originally on only Arkansas and Mississippi, I got
 really
  nice results. I then tried it on the entire US and had the missing
 counties
  problem. So I tried only California, and still have the same issue.
 
  I've tried other algorithms, but this has so far given me the detail I
 want
  - of course sans counties! Any thoughts?
 
  Thanks,
   - Chris
  ___
  grass-user mailing list
  grass-user@lists.osgeo.org
  http://lists.osgeo.org/mailman/listinfo/grass-user
 
 

___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] v.generalize issues

[Daniel Lee]

I've experienced problems with calling up vectors stored in the GRASS
database with QGIS, sometimes vectors simply are missing. Does the same
problem occur in the GRASS environment? Maybe it would help to try showing
the vectors in GRASS directly. If that work it means that QGIS is just
having a problem where it calls up the vectors from GRASS. My problem was
fixable by exporting the vectors as a shapefile and then loading them in
QGIS.

--
B.Sc. Daniel Lee
Geschäftsführung für Forschung und Entwicklung
ISIS - International Solar Information Solutions

Deutschhausstr. 10
35037 Marburg
Festnetz: +49 6421 379 6256
Mobil: +49 176 6127 7269
E-Mail: l...@isi-solutions.org
Web: http://www.isi-solutions.org
Am 30.06.2011 16:02 schrieb Christian Guirreri christ...@guirreri.com:
 I'm a brand new user to GIS - my goal right now is to heavily simplify
Tiger
 2010 county and distrct data, without producing gaps between boundaries.
 I've been testing this with v.generalize in grass via QuantumGIS - I've
had
 tons of issues with the Grass toolbox crashing, but have narrowed down to
 using the Hermite algorithm. While it works, I'm having some bizarre
issues
 with it. Apologies for the cross-post with the PostGIS mailing list.

 In the attached gif of California counties, from left to right, I have
used
 the following tolerance values with the Hermite algorithm:
 - original
 - 1.0
 - 0.08
 - 0.01
 - 0.1

 Why do counties disappear entirely as I decrease the tolerance?

 In the Grass Tools I choose the v.generalize function. I choose Boundary
as
 the feature type (though I've tried checking others, as well as all of
them
 and it doesn't seem to change anything). Everything else is default,
except
 for tolerance as notated above.

 When I tested this originally on only Arkansas and Mississippi, I got
really
 nice results. I then tried it on the entire US and had the missing
counties
 problem. So I tried only California, and still have the same issue.

 I've tried other algorithms, but this has so far given me the detail I
want
 - of course sans counties! Any thoughts?

 Thanks,
 - Chris
___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] v.generalize issues

[Christian Guirreri]

Understood. I actually started to try using some of the GRASS interface via
QGIS, but on some of the panels, such as v.generalize, I couldn't type in a
tolerance value. I'm on Win7 x64.

I'm going to try a fresh install of GRASS.



On Thu, Jun 30, 2011 at 10:52 AM, Daniel Lee l...@isi-solutions.org wrote:

 I've experienced problems with calling up vectors stored in the GRASS
 database with QGIS, sometimes vectors simply are missing. Does the same
 problem occur in the GRASS environment? Maybe it would help to try showing
 the vectors in GRASS directly. If that work it means that QGIS is just
 having a problem where it calls up the vectors from GRASS. My problem was
 fixable by exporting the vectors as a shapefile and then loading them in
 QGIS.

 --
 B.Sc. Daniel Lee
 Geschäftsführung für Forschung und Entwicklung
 ISIS - International Solar Information Solutions

 Deutschhausstr. 10
 35037 Marburg
 Festnetz: +49 6421 379 6256
 Mobil: +49 176 6127 7269
 E-Mail: l...@isi-solutions.org
 Web: http://www.isi-solutions.org
 Am 30.06.2011 16:02 schrieb Christian Guirreri christ...@guirreri.com:

  I'm a brand new user to GIS - my goal right now is to heavily simplify
 Tiger
  2010 county and distrct data, without producing gaps between boundaries.
  I've been testing this with v.generalize in grass via QuantumGIS - I've
 had
  tons of issues with the Grass toolbox crashing, but have narrowed down to
  using the Hermite algorithm. While it works, I'm having some bizarre
 issues
  with it. Apologies for the cross-post with the PostGIS mailing list.
 
  In the attached gif of California counties, from left to right, I have
 used
  the following tolerance values with the Hermite algorithm:
  - original
  - 1.0
  - 0.08
  - 0.01
  - 0.1
 
  Why do counties disappear entirely as I decrease the tolerance?
 
  In the Grass Tools I choose the v.generalize function. I choose Boundary
 as
  the feature type (though I've tried checking others, as well as all of
 them
  and it doesn't seem to change anything). Everything else is default,
 except
  for tolerance as notated above.
 
  When I tested this originally on only Arkansas and Mississippi, I got
 really
  nice results. I then tried it on the entire US and had the missing
 counties
  problem. So I tried only California, and still have the same issue.
 
  I've tried other algorithms, but this has so far given me the detail I
 want
  - of course sans counties! Any thoughts?
 
  Thanks,
  - Chris

___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] v.generalize issues

[Markus Metz]

On Thu, Jun 30, 2011 at 4:47 PM, Christian Guirreri
christ...@guirreri.com wrote:
 I'm on GRASS 6.4.1. QGIS 1.7.0

In this version, v.generalize is still broken. You would need a recent
version of GRASS 6.4.2. For Windows, you can get a recent version
here:

http://wingrass.fsv.cvut.cz/grass64/

Markus M


 On Thu, Jun 30, 2011 at 10:18 AM, Markus Metz
 markus.metz.gisw...@googlemail.com wrote:

 On Thu, Jun 30, 2011 at 4:01 PM, Christian Guirreri
 christ...@guirreri.com wrote:
  I'm a brand new user to GIS - my goal right now is to heavily simplify
  Tiger
  2010 county and distrct data, without producing gaps between boundaries.
  I've been testing this with v.generalize in grass via QuantumGIS - I've
  had
  tons of issues with the Grass toolbox crashing, but have narrowed down
  to
  using the Hermite algorithm. While it works, I'm having some bizarre
  issues
  with it. Apologies for the cross-post with the PostGIS mailing list.
 
  In the attached gif of California counties, from left to right, I have
  used
  the following tolerance values with the Hermite algorithm:
   - original
   - 1.0
   - 0.08
   - 0.01
   - 0.1
 
  Why do counties disappear entirely as I decrease the tolerance?
 
 This problem has been fixed in GRASS 6.4 only 2 weeks ago (June 13).
 Please update your GRASS version if possible.

 Markus M


  In the Grass Tools I choose the v.generalize function. I choose Boundary
  as
  the feature type (though I've tried checking others, as well as all of
  them
  and it doesn't seem to change anything). Everything else is default,
  except
  for tolerance as notated above.
 
  When I tested this originally on only Arkansas and Mississippi, I got
  really
  nice results. I then tried it on the entire US and had the missing
  counties
  problem. So I tried only California, and still have the same issue.
 
  I've tried other algorithms, but this has so far given me the detail I
  want
  - of course sans counties! Any thoughts?
 
  Thanks,
   - Chris
  ___
  grass-user mailing list
  grass-user@lists.osgeo.org
  http://lists.osgeo.org/mailman/listinfo/grass-user
 
 


___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] v.generalize issues

[Christian Guirreri]

Success! Thanks for making this easy for the total noob that I am.

On Thu, Jun 30, 2011 at 11:04 AM, Markus Metz 
markus.metz.gisw...@googlemail.com wrote:

 On Thu, Jun 30, 2011 at 4:47 PM, Christian Guirreri
 christ...@guirreri.com wrote:
  I'm on GRASS 6.4.1. QGIS 1.7.0
 
 In this version, v.generalize is still broken. You would need a recent
 version of GRASS 6.4.2. For Windows, you can get a recent version
 here:

 http://wingrass.fsv.cvut.cz/grass64/

 Markus M


  On Thu, Jun 30, 2011 at 10:18 AM, Markus Metz
  markus.metz.gisw...@googlemail.com wrote:
 
  On Thu, Jun 30, 2011 at 4:01 PM, Christian Guirreri
  christ...@guirreri.com wrote:
   I'm a brand new user to GIS - my goal right now is to heavily simplify
   Tiger
   2010 county and distrct data, without producing gaps between
 boundaries.
   I've been testing this with v.generalize in grass via QuantumGIS -
 I've
   had
   tons of issues with the Grass toolbox crashing, but have narrowed down
   to
   using the Hermite algorithm. While it works, I'm having some bizarre
   issues
   with it. Apologies for the cross-post with the PostGIS mailing list.
  
   In the attached gif of California counties, from left to right, I have
   used
   the following tolerance values with the Hermite algorithm:
- original
- 1.0
- 0.08
- 0.01
- 0.1
  
   Why do counties disappear entirely as I decrease the tolerance?
  
  This problem has been fixed in GRASS 6.4 only 2 weeks ago (June 13).
  Please update your GRASS version if possible.
 
  Markus M
 
 
   In the Grass Tools I choose the v.generalize function. I choose
 Boundary
   as
   the feature type (though I've tried checking others, as well as all of
   them
   and it doesn't seem to change anything). Everything else is default,
   except
   for tolerance as notated above.
  
   When I tested this originally on only Arkansas and Mississippi, I got
   really
   nice results. I then tried it on the entire US and had the missing
   counties
   problem. So I tried only California, and still have the same issue.
  
   I've tried other algorithms, but this has so far given me the detail I
   want
   - of course sans counties! Any thoughts?
  
   Thanks,
- Chris
   ___
   grass-user mailing list
   grass-user@lists.osgeo.org
   http://lists.osgeo.org/mailman/listinfo/grass-user
  
  
 
 

___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] v.generalize crash

[Markus Metz]

On Tue, May 4, 2010 at 6:52 PM, Paolo Cavallini cavall...@faunalia.it wrote:
 Nikos Alexandris ha scritto:

 Does it crash immediately or after a long time? It works for me in a small
 file (boundaries though, not lines) and takes forever (still running after
 15min in a big file containing contours) but no crash.

 It is a memory error: when filled up it crashes. Obviously this does not 
 happen with
 simpler layers.

From the manual: Snakes Algorithm is (asymptotically) the slowest
among the algorithms presented above. Also, it requires quite a lot of
memory. This means that it is not very efficient for maps with the
lines consisting of many segments.

Try another smoothing algorithm, e.g. chaiken.

HTH,

Markus M
___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] v.generalize crash

[Paolo Cavallini]

Markus Metz ha scritto:

 Try another smoothing algorithm, e.g. chaiken.

Thanks. Yes, Chaiken and Hermite work. However, IMHO it should not crash, just 
stop.
All the best.
-- 
Paolo Cavallini: http://www.faunalia.it/pc
___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] v.generalize crash

[Markus Metz]

On Wed, May 5, 2010 at 10:49 AM, Paolo Cavallini cavall...@faunalia.it wrote:
 Markus Metz ha scritto:

 Try another smoothing algorithm, e.g. chaiken.

 Thanks. Yes, Chaiken and Hermite work. However, IMHO it should not crash, 
 just stop.

Right. I'll fix it, it should abort with fatal error: Out of memory.

Markus M
___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] v.generalize crash

[Paolo Cavallini]

Markus Metz ha scritto:
 Right. I'll fix it, it should abort with fatal error: Out of memory.

Thanks a lot for this.
-- 
Paolo Cavallini: http://www.faunalia.it/pc
___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] v.generalize for area boundaries?

[Markus Neteler]

On Tue, Feb 17, 2009 at 7:35 AM, Hamish hamis...@yahoo.com wrote:
 Hi,

 I am following the v.generalize tutorial at
  http://users.ox.ac.uk/~orie1848/tutorial.html
   (we should move that to the wiki before it disappears)

Here are converters to Mediawiki syntax:

http://www.jtidy.de/
http://diberri.dyndns.org/wikipedia/html2wiki/index.html

Additionally, the screenshots need to be uploaded.

Markus
___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] v.generalize for area boundaries?

[Wolf Bergenheim]

On 17.02.2009 08:35, Hamish wrote:
 Hi, 
 
 I am following the v.generalize tutorial at
   http://users.ox.ac.uk/~orie1848/tutorial.html
(we should move that to the wiki before it disappears)
 but it doesn't say much about working with areas beyond removing small
 ones.

That page and the images exists in the svn too. How does one go about
adding extra manual pages? Or perhaps we could integrate it into the
manual page itself?

 
 I have a vector area which has a very steppy boundary, like from
 r.to.vect with a sawtooth pattern at the cell edges. I want to run a
 smoothing filter over it to get rid of the jaggy bits.
 
 No matter what method I try my output map is always the same as the input
 map, no vertices are created or destroyed.
 
 any ideas how to do this?  I know about 'v.clean tool=prune' and Markus
 Metz's topology-preserving v.simplify (psst- add it to wiki addons) but
 I'd like to learn more about how to use v.generalize.

How large is the cell you exported from? First you have to use a
smoothing operator. When you are done with the smoothing you should use
the douglas method to reduce the number of points.

I'll see if I can whip up an example for you.

--Wolf

-- 

:3 ) Wolf Bergenheim ( 8:


___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] v.generalize for area boundaries?

[FAROUX STEPHANIE]

Hi,
Is it possible with grass to replace missing values in a map by 
surrounding values by a sort of interpolation, knowing that sometimes 
many missing values are numerous one near another? If yes, how?

Thank you very much
Stéphanie
___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] v.generalize for area boundaries?

[ivan marchesini]

Hi..
please try r.neighbors

cheers..
Ivan


Il giorno mar, 17/02/2009 alle 18.00 +0100, FAROUX STEPHANIE ha scritto:
 Hi,
 Is it possible with grass to replace missing values in a map by 
 surrounding values by a sort of interpolation, knowing that sometimes 
 many missing values are numerous one near another? If yes, how?
 Thank you very much
 Stéphanie
 ___
 grass-user mailing list
 grass-user@lists.osgeo.org
 http://lists.osgeo.org/mailman/listinfo/grass-user
 
-- 
Ti prego di cercare di non inviarmi files .dwg, .doc, .xls, .ppt.
Preferisco formati liberi.
Please try to avoid to send me .dwg, .doc, .xls, .ppt files.
I prefer free formats.
http://it.wikipedia.org/wiki/Formato_aperto
http://en.wikipedia.org/wiki/Open_format

Ivan Marchesini
Department of Civil and Environmental Engineering
University of Perugia
Via G. Duranti 93/a 
06125
Perugia (Italy)
Socio fondatore GFOSS Geospatial Free and Open Source Software 
http://www.gfoss.it
e-mail: marches...@unipg.it
ivan.marches...@gmail.com
tel: +39(0)755853760
fax (university): +39(0)755853756
fax (home): +39(0)5782830887
jabber: geoiva...@jabber.org

___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] v.generalize for area boundaries?

[Markus Neteler]

On Tue, Feb 17, 2009 at 7:03 PM, ivan marchesini marches...@unipg.it wrote:
 Hi..
 please try r.neighbors

r.fillnulls is another option:
http://grass.osgeo.org/grass64/manuals/html64_user/r.fillnulls.html

Markus

 cheers..
 Ivan


 Il giorno mar, 17/02/2009 alle 18.00 +0100, FAROUX STEPHANIE ha scritto:
 Hi,
 Is it possible with grass to replace missing values in a map by
 surrounding values by a sort of interpolation, knowing that sometimes
 many missing values are numerous one near another? If yes, how?
 Thank you very much
 Stéphanie
___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] v.generalize / reanimation of dead lines ?

[Benjamin Ducke]

Further on this subject, I tried v.generalize on the North Carolina
sample dataset and ran into the same attempt to read dead line error
with the boundary_county areal layer. Very small threshold settings
(  1.0 ) worked OK, but obviously, there is only a minimal
simplification to be gained from this. Any threshold setting around 10.0
resulted in the above error not more than 10% through the data.
I tried all different algorithms and did not have better success with
any of them.

Best,

Ben

Markus Metz wrote:

Hi Peter,

as an interim solution you might try an alternative to v.generalize that
I called v.simplify, available here:
http://markus.metz.giswork.googlepages.com/line_simplification.tar.gz
The module works for me so far, but I still discover strange behaviour
now and then. I developed that module specifically for areas, before
v.generalize was available.
Differences to v.generalize are that this module only
simplifies/generalizes lines/boundaries, smoothing is not available, and
only the Douglas-Peucker algorithm is implemented.
The appropriate level of topology is always maintained, centroids are
always kept and never altered. Boundaries are not simplified if this
would result in area deletion or changed centroid attachment. All the
work is done with a temporary vector, and after simplifying the
temporary vector, only alive lines are copied to the final output vector
(no dead lines, smaller file size).  v.simplify can also delete
duplicate points only, i.e. of two adjacent vertices in a line/boundary
with identical coordinates one is removed. This should be done before
simplification in v.simplify.
As far as I understand the source code v.generalize discards all
centroids and builds them anew at the end, area topology is not
maintained during simplification.
This is not meant to be competition for v.generalize, just some ideas on
how to avoid dead lines in the output vector and how to preserve all
areas/centroids, with their original category ID.

Markus
 


Peter Löwe wrote:

Hi,

I second the unclean deletion theory as I encountered this phenomenon before. Also, when generalizing areas which are 
attached to each other, while the overall appearance of the generalized borderlines is ok, more than 50 % of the 
areas cease to exist, which needs further investigation: Apart from dead lines we're dealing with area 
mutilation and centroid abduction!

The (topological) truth is somewhere out there,
Peter


 Original-Nachricht 
  

Datum: Fri, 29 Aug 2008 19:17:46 +0300
Von: Wolf Bergenheim [EMAIL PROTECTED]
An: [EMAIL PROTECTED]
CC: grass-user@lists.osgeo.org, [EMAIL PROTECTED], [EMAIL PROTECTED], Daniel Bundala 
[EMAIL PROTECTED]
Betreff: Re: [GRASS-user] v.generalize / reanimation of dead lines ?

  

On 29.08.2008 18:54, Dylan Beaudette wrote:


I have noticed this behavior as well when using v.generalize. I will try
  
and 


dig up the exact situation that caused it-- but I am pretty sure that
  
the 


initial linework was correct = unclean deletion.

  

That is very much possible. This module is still quite young and hasn't
gone through a lot of live testing yet. Dylan, I'd be vey interested in
this, if you can give me a simple case where it goes wrong.

--Wolf

Adding Daniel Bundala to the Cc list, as he wrote it last summer.

--

:3 ) Wolf Bergenheim ( 8:

  

___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user





--
Benjamin Ducke
Senior Applications Support and Development Officer

Oxford Archaeology Ltd
Janus House
Osney Mead
OX2 0ES
Oxford, U.K.

Tel: +44 (0)1865 263 800 (switchboard)
Tel: +44 (0)1865 980 758 (direct)
Fax :+44 (0)1865 793 496
[EMAIL PROTECTED]




--
Files attached to this email may be in ISO 26300 format (OASIS Open Document 
Format). If you have difficulty opening them, please visit http://iso26300.info 
for more information.

___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] v.generalize / reanimation of dead lines ?

[Peter Löwe]

Hi,

I second the unclean deletion theory as I encountered this phenomenon before. 
Also, when generalizing areas which are attached to each other, while the 
overall appearance of the generalized borderlines is ok, more than 50 % of 
the areas cease to exist, which needs further investigation: Apart from dead 
lines we're dealing with area mutilation and centroid abduction!

The (topological) truth is somewhere out there,
Peter


 Original-Nachricht 
 Datum: Fri, 29 Aug 2008 19:17:46 +0300
 Von: Wolf Bergenheim [EMAIL PROTECTED]
 An: [EMAIL PROTECTED]
 CC: grass-user@lists.osgeo.org, [EMAIL PROTECTED], [EMAIL PROTECTED], Daniel 
 Bundala [EMAIL PROTECTED]
 Betreff: Re: [GRASS-user] v.generalize / reanimation of dead lines ?

 On 29.08.2008 18:54, Dylan Beaudette wrote:
  
  I have noticed this behavior as well when using v.generalize. I will try
 and 
  dig up the exact situation that caused it-- but I am pretty sure that
 the 
  initial linework was correct = unclean deletion.
  
 
 That is very much possible. This module is still quite young and hasn't
 gone through a lot of live testing yet. Dylan, I'd be vey interested in
 this, if you can give me a simple case where it goes wrong.
 
 --Wolf
 
 Adding Daniel Bundala to the Cc list, as he wrote it last summer.
 
 -- 
 
 :3 ) Wolf Bergenheim ( 8:

-- 
Dr. Peter Löwe
[EMAIL PROTECTED]





Ist Ihr Browser Vista-kompatibel? Jetzt die neuesten 
Browser-Versionen downloaden: http://www.gmx.net/de/go/browser
___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] v.generalize / reanimation of dead lines ?

[Markus Metz]

Hi Peter,

as an interim solution you might try an alternative to v.generalize that
I called v.simplify, available here:
http://markus.metz.giswork.googlepages.com/line_simplification.tar.gz
The module works for me so far, but I still discover strange behaviour
now and then. I developed that module specifically for areas, before
v.generalize was available.
Differences to v.generalize are that this module only
simplifies/generalizes lines/boundaries, smoothing is not available, and
only the Douglas-Peucker algorithm is implemented.
The appropriate level of topology is always maintained, centroids are
always kept and never altered. Boundaries are not simplified if this
would result in area deletion or changed centroid attachment. All the
work is done with a temporary vector, and after simplifying the
temporary vector, only alive lines are copied to the final output vector
(no dead lines, smaller file size).  v.simplify can also delete
duplicate points only, i.e. of two adjacent vertices in a line/boundary
with identical coordinates one is removed. This should be done before
simplification in v.simplify.
As far as I understand the source code v.generalize discards all
centroids and builds them anew at the end, area topology is not
maintained during simplification.
This is not meant to be competition for v.generalize, just some ideas on
how to avoid dead lines in the output vector and how to preserve all
areas/centroids, with their original category ID.

Markus
 

Peter Löwe wrote:
 Hi,

 I second the unclean deletion theory as I encountered this phenomenon 
 before. Also, when generalizing areas which are attached to each other, while 
 the overall appearance of the generalized borderlines is ok, more than 50 % 
 of the areas cease to exist, which needs further investigation: Apart from 
 dead lines we're dealing with area mutilation and centroid abduction!

 The (topological) truth is somewhere out there,
 Peter


  Original-Nachricht 
   
 Datum: Fri, 29 Aug 2008 19:17:46 +0300
 Von: Wolf Bergenheim [EMAIL PROTECTED]
 An: [EMAIL PROTECTED]
 CC: grass-user@lists.osgeo.org, [EMAIL PROTECTED], [EMAIL PROTECTED], Daniel 
 Bundala [EMAIL PROTECTED]
 Betreff: Re: [GRASS-user] v.generalize / reanimation of dead lines ?
 

   
 On 29.08.2008 18:54, Dylan Beaudette wrote:
 
 I have noticed this behavior as well when using v.generalize. I will try
   
 and 
 
 dig up the exact situation that caused it-- but I am pretty sure that
   
 the 
 
 initial linework was correct = unclean deletion.

   
 That is very much possible. This module is still quite young and hasn't
 gone through a lot of live testing yet. Dylan, I'd be vey interested in
 this, if you can give me a simple case where it goes wrong.

 --Wolf

 Adding Daniel Bundala to the Cc list, as he wrote it last summer.

 -- 

 :3 ) Wolf Bergenheim ( 8:
 

   
___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] v.generalize / reanimation of dead lines ?

[Daniel Bundala]

On Mon, Sep 1, 2008 at 3:55 PM, Markus Metz [EMAIL PROTECTED] wrote:
 As far as I understand the source code v.generalize discards all
 centroids and builds them anew at the end, area topology is not
 maintained during simplification.

Dear all,
That is correct. The intention was that the generalization may change
the shape of the lines and boundaries so that the original centroid
may no longer be inside the area and so the centroids need to be
recalculated. Also, the generalization may remove some lines/areas and
hence alter the topology of the map. So it also needs to be
recalculated.

Daniel
___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] v.generalize / reanimation of dead lines ?

[Dylan Beaudette]

On Thursday 28 August 2008, Hamish wrote:
 Peter:
   when trying to derive a continous coast_line_ from the
   GSHHS data set. For this, the data was reduced using
   v.generalize (douglas, threshold=0.001).
  
   Since the reduced coastline consists of several lines (with
   individual categories), I tried v.build to snap them
   together into a (hopefully) one long and friendly line.
  
   Alas, v.build throws:
  
   V2_read_line_nat(): Attempt to read dead line 26199
  
   Any advice how to reanimate the dead lines ??

 two ideas:
 before:  v.build.polylines
 after:   v.extract list=1-99

 do lines actually seem to be missing or was it unclean deletion?


 Hamish


I have noticed this behavior as well when using v.generalize. I will try and 
dig up the exact situation that caused it-- but I am pretty sure that the 
initial linework was correct = unclean deletion.

Cheers,

Dylan


-- 
Dylan Beaudette
Soil Resource Laboratory
http://casoilresource.lawr.ucdavis.edu/
University of California at Davis
530.754.7341
___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] v.generalize / reanimation of dead lines ?

[Wolf Bergenheim]

On 29.08.2008 18:54, Dylan Beaudette wrote:
 
 I have noticed this behavior as well when using v.generalize. I will try and 
 dig up the exact situation that caused it-- but I am pretty sure that the 
 initial linework was correct = unclean deletion.
 

That is very much possible. This module is still quite young and hasn't
gone through a lot of live testing yet. Dylan, I'd be vey interested in
this, if you can give me a simple case where it goes wrong.

--Wolf

Adding Daniel Bundala to the Cc list, as he wrote it last summer.

-- 

:3 ) Wolf Bergenheim ( 8:

___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] v.generalize / reanimation of dead lines ?

[Wolf Bergenheim]

Peter,

Have you tried to v.build snap the lined before you generalize them?

--Wolf

On 28.08.2008 19:04, [EMAIL PROTECTED] wrote:
 Hi,
 
 when trying to derive a continous coast_line_ from the GSHHS data
 set. For this, the data was reduced using v.generalize (douglas,
 threshold=0.001).
 
 Since the reduced coastline consists of several lines (with
 individual categories), I tried v.build to snap them together into a
 (hopefully) one long and friendly line.
 
 Alas, v.build throws:
 
 V2_read_line_nat(): Attempt to read dead line 26199
 
 Any advice how to reanimate the dead lines ??
 
 
 Peter

-- 

:3 ) Wolf Bergenheim ( 8:

___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] v.generalize for polygons?

[Wolf Bergenheim]

Hamish,

I played around with the map you gave me, and I think I found a way 
around the problem (though I'm still not very sure about what the real 
problem is, perhaps too many small areas or too many shorter segments 
(the boundary seems to be built of a number of smaller lines). I'll try 
to create another problematic map.

Anyway, here is how I was able to generalize it:

# First I fuse the short segments into one long boundary
v.build.polylines --o input=rc_merge_coast3 output=fused
# Then I clean away some (relatively) small islands
v.clean --o in=fused out=clean tool=rmarea thresh=50.0
# finally generalize
v.generalize -r --o input=clean output=gen method=douglas_reduction 
reduction=20


The generalized map contains about 20% of the original points.

--Wolf

On 28.02.2008 06:16, Hamish wrote:

Hamish:

I have a high-res vector area map of regional districts which I
wish to generalize. I am having trouble with finding the correct
method in v.generalize to use. Currently every thing I try tends
to break the area topology and leave only a portion of the now-
open boundary.


I have now tried with a related vector, linked below, and it worked
(very!) nicely for that. But it fails with a derivative vector map.
v.digit shows no problems with topography.

Wolf:

What methods did you try?


many of them.. mainly douglas with a number of threshold values.

What exact commands have you tried that fail? 


at the simplest:   v.generalize in= out=
but some areas are missing.



Can you share the problematic map? (you can email it to me directly)


sure,

starting with:
http://www.stats.govt.nz/statistics-by-area/regional-statistics/geography-mapping/download-digital-boundaries.htm
-- Census based NZMG 2006 (37mb shapefile .zip)

I am looking at regional boundaries (RC) from REGC06_LV2.shp

this map generalizes nicely, but it includes the 12 nautical mile
territorial buffer around the coastline. When I overlay that map with a
detailed coastline is when I see the problem.

I'll send a sample of the v.overlay output off-list.


v.generalize does preserve nodes, and as long as the input map is 
topologically correct so should the output map be.


ok. (confirmed, it does a very nice job simplifying the above
shapefile)


Perhaps your threshold is way off?


Possible, as I am just learning. But I did try a number of ranges and
slowly increase. All would be ok for slight generalization then big
breakage.

e.g. it has a big jump between thresh=0.4865 and 0.487

Good:
   v.generalize in=rc_merge_coast3 out=rc_gen thresh=0.4865 --o
   ...
   Number of vertices was reduced from 569815 to 521969 [91%]

Bad:
   v.generalize in=rc_merge_coast3 out=rc_gen thresh=0.487 --o
   ...
   Number of vertices was reduced from 569815 to 336380 [59%]


Daniel:

However, there is a flag(-r?) which prevents the module from removing
them.


Flags:
  -c   Copy attributes
  -r   Remove lines and areas smaller than threshold



thanks,
Hamish




  

Looking for last minute shopping deals?  
Find them fast with Yahoo! Search.  http://tools.search.yahoo.com/newsearch/category.php?category=shopping




--

:3 ) Wolf Bergenheim ( 8:

___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] v.generalize for polygons?

[Daniel Bundala]

Hi,

I am not sure that I understand correcly what you are trying to do,
but as far as I remember, the boundaries are generalized as polylines.
That is, if the boundary contains some corners, they might be removed
which might create the holes. Also, ovesimplified lines are removed
by default. These are the lines that are shorter than the threshold.
However, there is a flag(-r?) which prevents the module from removing
them.

Daniel

On Wed, Feb 27, 2008 at 2:19 AM, Hamish [EMAIL PROTECTED] wrote:
 Hi,

  I have a high-res vector area map of regional districts which I wish to 
 generalize. I am having trouble with finding the correct method in 
 v.generalize to use. Currently every thing I try tends to break the area 
 topology and leave only a portion of the now-open boundary.

  i.e. it should preserve nodes, only generalize (remove) non-node vertices.
  We can assume there is only a single boundary line between areas, so there 
 shouldn't be topological issues. (or at least only in tiny corner cases where 
 the new generalized line overlaps another feature, but that should be easy to 
 fix with v.clean)

  the idea is to simplify the map before running v.extrude to get something 
 like this:
  http://grass.osgeo.org/grass60/screenshots/images/inc_employ_usa_2002.jpg

  but without the 600,000 extruded faces created from every little twist in 
 the coastline.


  Hamish
  ___
  grass-user mailing list
  grass-user@lists.osgeo.org
  http://lists.osgeo.org/mailman/listinfo/grass-user

___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] v.generalize for polygons?

[Hamish]

Hamish:
  I have a high-res vector area map of regional districts which I
  wish to generalize. I am having trouble with finding the correct
  method in v.generalize to use. Currently every thing I try tends
  to break the area topology and leave only a portion of the now-
  open boundary.

I have now tried with a related vector, linked below, and it worked
(very!) nicely for that. But it fails with a derivative vector map.
v.digit shows no problems with topography.

Wolf:
 What methods did you try?

many of them.. mainly douglas with a number of threshold values.

 What exact commands have you tried that fail? 

at the simplest:   v.generalize in= out=
but some areas are missing.


 Can you share the problematic map? (you can email it to me directly)

sure,

starting with:
http://www.stats.govt.nz/statistics-by-area/regional-statistics/geography-mapping/download-digital-boundaries.htm
-- Census based NZMG 2006 (37mb shapefile .zip)

I am looking at regional boundaries (RC) from REGC06_LV2.shp

this map generalizes nicely, but it includes the 12 nautical mile
territorial buffer around the coastline. When I overlay that map with a
detailed coastline is when I see the problem.

I'll send a sample of the v.overlay output off-list.


 v.generalize does preserve nodes, and as long as the input map is 
 topologically correct so should the output map be.

ok. (confirmed, it does a very nice job simplifying the above
shapefile)

 Perhaps your threshold is way off?

Possible, as I am just learning. But I did try a number of ranges and
slowly increase. All would be ok for slight generalization then big
breakage.

e.g. it has a big jump between thresh=0.4865 and 0.487

Good:
   v.generalize in=rc_merge_coast3 out=rc_gen thresh=0.4865 --o
   ...
   Number of vertices was reduced from 569815 to 521969 [91%]

Bad:
   v.generalize in=rc_merge_coast3 out=rc_gen thresh=0.487 --o
   ...
   Number of vertices was reduced from 569815 to 336380 [59%]


Daniel:
 However, there is a flag(-r?) which prevents the module from removing
 them.

Flags:
  -c   Copy attributes
  -r   Remove lines and areas smaller than threshold



thanks,
Hamish




  

Looking for last minute shopping deals?  
Find them fast with Yahoo! Search.  
http://tools.search.yahoo.com/newsearch/category.php?category=shopping

___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user