[GRASS-user] issues with r.horizon incombination with multiprocessing

[Tinsley Wolfs]

Hi list,

Based on a discussion in the list archive, I copied and edited a script that 
parallelizes the r.horizon module across multiple cores 
(link).
 I have updated this script to work with the current module parameter naming 
schemes where required. Additionally I added a for loop to the main() function, 
as I want it to do the r.horizon module for several areas sequentially (setting 
g.region, etc...).

I have several issues:

1. Each time it creates the workers for the last r.horizon angles for a list 
item, it also creates the workers for the first angles of the next item in the 
list. But it should finish the first workers, copy the raster from the 
temporary mapsets to the current mapset and delete the temporary mapsets before 
creating new workers and starting the process again.
2. It doesn't calculate the last angles of the last area part of the list that 
it loops over. Though I think this is because of the first issue which is that 
the order of execution is off.

Why doesn't it follow the correct order as per the script but move on to the 
next step before finishing the last? I run the script from the simple python 
editor in Grass 7.2.0.
#!/usr/bin/env python

import sys, os, shutil, re
import datetime as dt
import multiprocessing as mp
import subprocess
import atexit

gisbase = os.environ['GISBASE'] = r'C:\\Program Files\\GRASS GIS 7.2.0\\'
gisdbase = r'C:\data\grass'
sys.path.append(os.path.join(os.environ['GISBASE'], "etc", "python"))

import grass.script as grass
import grass.script.setup as gsetup

# code to create list with name of vectors to iterate over
className = "bykerne" + "Net"
mapName = className + '@Solar'  # counting layers (sample clusters)
result = grass.read_command('v.info',   # in vector file
flags = 'e',
map = mapName)

result = str.split(result, '\n')
layerCount = result[12] # dblink_num = layers 
layerCount = str.split(layerCount, '=')
layerCount = layerCount[1]  # number of layers
layerCount = int(layerCount)

layerN = 1  # start layer count
outputName = className + '_' + str(layerN - 1)
vectorList = grass.read_command('g.list', type = 'vector', mapset = 'Solar')
vectorList = str.split(vectorList, '\n')
vectorList = ''.join(vectorList)
vectorList = str.split(vectorList, '\r')
vectorList.pop(0)
vectorList.pop(0)
vectorList.pop(-1) # final vector list

# parameters
location = 'Denmark'
mhorizon = 'Solar'  
maxDistance = '500' 
hstep = 15  
distance = '1.0'

# start of function definitions:

def worker_horizon(cpu,demh,azimuth, outputNameS):

# Setup Grass Environment, including temporary mapset

mtemp = 'temp'+str(cpu).zfill(2)

gsetup.init(gisbase, gisdbase, location, mtemp)

grass.run_command("g.region",vector = outputNameS +"@Solar")



# map names

prefhor = demh+'hor'

demh = demh+'@Solar'



# run r.horizon for one azimuth angle

azi = str(azimuth)

grass.run_command("r.horizon", elevation=demh, \

  direction=azi, maxdistance=maxDistance, \

  output=prefhor, dist=distance, overwrite = True)



# rename horizon map to proper azimuth name

az = str(azimuth).zfill(3)

tempmap = prefhor+'_0'

horizonmap = prefhor+'_'+az

cmd = tempmap+","+horizonmap

grass.run_command("g.rename",rast=cmd,overwrite = True)



def create_temp(cores):

gsetup.init(gisbase, gisdbase, location, 'PERMANENT')

for count in range(0,cores,1):

temp = 'temp'+str(count).zfill(2)

temp_path = gisdbase+'/'+location+'/'+temp

if(os.path.exists(temp_path) == False):

grass.run_command("g.mapset",flags="c", mapset=temp, 
quiet=1)

print(temp+" mapset created.")

else:

print(temp+" mapset already exists. skipping...")



def remove_temp(cores):

# setup is in the target mapset, not temp mapsets

gsetup.init(gisbase, gisdbase, location, mhorizon)

# Delete the temp mapset

for count in range(0,cores):

mapset_temp = 'temp'+str(count).zfill(2)

grass.run_command("g.mapsets", mapset=mapset_temp, operation = 
'remove')

temp_path = gisdbase+'/'+location+'/'+mapset_temp

shutil.rmtree(temp_path)

if os.path.exists(temp_path):
shutil.rmtree(temp_path)


def copy_mapset(mapset_from,mapset_to,regfilter,overwrite):

# list contents of temp mapset

grass.run_command("g.mapset", mapset=mapset_from, quiet=1)

raster_list = 

Re: [GRASS-user] New peer-review article about dynamic flood simulation with GRASS and Itzï

[Anna Petrášová]

On Tue, May 2, 2017 at 12:47 PM, Laurent C.  wrote:
> Hello all,
>
> A new open-access, peer-reviewed journal article has been published
> about the use of GRASS and Itzï for the dynamic simulation of floods:
>
> Courty, L. G., Pedrozo-Acuña, A., and Bates, P. D.: Itzï (version
> 17.1): an open-source, distributed GIS model for dynamic flood
> simulation, Geosci. Model Dev., 10, 1835-1847,
> doi:10.5194/gmd-10-1835-2017, 2017.
>
> URL: http://www.geosci-model-dev.net/10/1835/2017/

Congratulations! This is a great and much needed addition to GRASS GIS!

Thank you,
Anna

>
> I hope you find it interesting.
>
> Best regards,
> Laurent Courty
> ___
> grass-user mailing list
> grass-user@lists.osgeo.org
> https://lists.osgeo.org/mailman/listinfo/grass-user
___
grass-user mailing list
grass-user@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] CHELSA climate data set

[Veronica Andreo]

Hi Helli,

Yes, if you want the global map the -a flag plus the relaxed conditions for
the extent of global maps are fine...

However, I wanted to also only import the region i'm working on and make
use of the -r flag in r.in.gdal... then the problems start... no
combination of -a, raster or align parameters in r.region give me the
correct region (either the resolution is wrong or the extent). My solution
so far is import the global map with r.in.gdal -a, then use r.mapcalc to
get the region I need... any other suggestion about better workflow is more
than welcome :)

best,
Vero

2017-05-02 21:45 GMT+02:00 Helmut Kudrnovsky :

> Veronica Andreo wrote
> > Hi :)
> >
> > JFYI, CHELSA v1.2 is out
> >
> > 2017-02-07 15:43 GMT+01:00 Markus Neteler 
>
> > neteler@
>
> > :
> >
> >> On Tue, Feb 7, 2017 at 3:32 PM, Markus Metz
> >> 
>
> > markus.metz.giswork@
>
> >  wrote:
> >> [...]
> >> > Compared to GMTED, CHELSA extents further south to 90:0:0.5S which
> >> conforms
> >> > to the GMTED grid geometry. For CHELSA, however gdalinfo reports
> >> > Pixel Size = (0.0083330,-0.0083330)
> >> > while for GMTED, gdalinfo reports
> >> > Pixel Size = (0.008,-0.008)
> >> >
> >> > The reason for the slightly smaller pixel size is most probably
> reduced
> >> > floating point precision during creation of the CHELSA data.
> >>
> >> from personal comm.:
> >> They are currently investigating where the precision is cut in their
> >> workflow (the upcoming V1.2 shall come with a related correction).
> >>
> >
> > Unfortunately they have not corrected the precision issue...
> >
> > gdalinfo CHELSA_temp_2_1979-2013_V1.2_land.tif
> > Driver: GTiff/GeoTIFF
> > Files: CHELSA_temp_2_1979-2013_V1.2_land.tif
> > Size is 43200, 20880
> > [...]
> > Origin = (-180.00013850017,83.999860415150010)
> > Pixel Size = (0.0083330,-0.0083330)
> >
> > Corner Coordinates:
> > Upper Left  (-180.0001389,  83.9998604) (180d 0' 0.50"W, 83d59'59.50"N)
> > Lower Left  (-180.0001389, -90.0001389) (180d 0' 0.50"W, 90d 0' 0.50"S)
> > Upper Right ( 179.9998597,  83.9998604) (179d59'59.49"E, 83d59'59.50"N)
> > Lower Right ( 179.9998597, -90.0001389) (179d59'59.49"E, 90d 0' 0.50"S)
> > Center  (  -0.0001396,  -3.0001392) (  0d 0' 0.50"W,  3d 0' 0.50"S)
> > Band 1 Block=43200x1 Type=Float32, ColorInterp=Gray
> >   NoData Value=-9
> >
> > best,
> > Vero
>
> yes, AFAIK they're still working on it.
>
> tested here ver.1.2 with r.in.gdal trunk and
>
> -a
> Auto-adjustment for lat/lon
> Attempt to fix small precision errors in resolution and extents
>
> it seems to give good results.
>
>
>
>
> -
> best regards
> Helmut
> --
> View this message in context: http://osgeo-org.1560.x6.
> nabble.com/CHELSA-climate-data-set-tp5284115p5319124.html
> Sent from the Grass - Users mailing list archive at Nabble.com.
> ___
> grass-user mailing list
> grass-user@lists.osgeo.org
> https://lists.osgeo.org/mailman/listinfo/grass-user
>
___
grass-user mailing list
grass-user@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] CHELSA climate data set

[Helmut Kudrnovsky]

Veronica Andreo wrote
> Hi :)
> 
> JFYI, CHELSA v1.2 is out
> 
> 2017-02-07 15:43 GMT+01:00 Markus Neteler 

> neteler@

> :
> 
>> On Tue, Feb 7, 2017 at 3:32 PM, Markus Metz
>> 

> markus.metz.giswork@

>  wrote:
>> [...]
>> > Compared to GMTED, CHELSA extents further south to 90:0:0.5S which
>> conforms
>> > to the GMTED grid geometry. For CHELSA, however gdalinfo reports
>> > Pixel Size = (0.0083330,-0.0083330)
>> > while for GMTED, gdalinfo reports
>> > Pixel Size = (0.008,-0.008)
>> >
>> > The reason for the slightly smaller pixel size is most probably reduced
>> > floating point precision during creation of the CHELSA data.
>>
>> from personal comm.:
>> They are currently investigating where the precision is cut in their
>> workflow (the upcoming V1.2 shall come with a related correction).
>>
> 
> Unfortunately they have not corrected the precision issue...
> 
> gdalinfo CHELSA_temp_2_1979-2013_V1.2_land.tif
> Driver: GTiff/GeoTIFF
> Files: CHELSA_temp_2_1979-2013_V1.2_land.tif
> Size is 43200, 20880
> [...]
> Origin = (-180.00013850017,83.999860415150010)
> Pixel Size = (0.0083330,-0.0083330)
> 
> Corner Coordinates:
> Upper Left  (-180.0001389,  83.9998604) (180d 0' 0.50"W, 83d59'59.50"N)
> Lower Left  (-180.0001389, -90.0001389) (180d 0' 0.50"W, 90d 0' 0.50"S)
> Upper Right ( 179.9998597,  83.9998604) (179d59'59.49"E, 83d59'59.50"N)
> Lower Right ( 179.9998597, -90.0001389) (179d59'59.49"E, 90d 0' 0.50"S)
> Center  (  -0.0001396,  -3.0001392) (  0d 0' 0.50"W,  3d 0' 0.50"S)
> Band 1 Block=43200x1 Type=Float32, ColorInterp=Gray
>   NoData Value=-9
> 
> best,
> Vero

yes, AFAIK they're still working on it.

tested here ver.1.2 with r.in.gdal trunk and

-a
Auto-adjustment for lat/lon
Attempt to fix small precision errors in resolution and extents

it seems to give good results.




-
best regards
Helmut
--
View this message in context: 
http://osgeo-org.1560.x6.nabble.com/CHELSA-climate-data-set-tp5284115p5319124.html
Sent from the Grass - Users mailing list archive at Nabble.com.
___
grass-user mailing list
grass-user@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] Re-project a spacio-temporal dataset

[Veronica Andreo]

Hi Laurent,

2017-05-02 20:57 GMT+02:00 Laurent C. :

> Hello,
>
> Is there a module that simplifies the reprojection of a complete STDS
> from a location to another?
>

Not that I know... t.rast.import/export work with the same projection,
AFAIU.

However, there's g.projall add-on :)

HTH
Vero
___
grass-user mailing list
grass-user@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-user

[GRASS-user] Re-project a spacio-temporal dataset

[Laurent C.]

Hello,

Is there a module that simplifies the reprojection of a complete STDS
from a location to another?

Regards,
Laurent
___
grass-user mailing list
grass-user@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] CHELSA climate data set

[Veronica Andreo]

Hi :)

JFYI, CHELSA v1.2 is out

2017-02-07 15:43 GMT+01:00 Markus Neteler :

> On Tue, Feb 7, 2017 at 3:32 PM, Markus Metz
>  wrote:
> [...]
> > Compared to GMTED, CHELSA extents further south to 90:0:0.5S which
> conforms
> > to the GMTED grid geometry. For CHELSA, however gdalinfo reports
> > Pixel Size = (0.0083330,-0.0083330)
> > while for GMTED, gdalinfo reports
> > Pixel Size = (0.008,-0.008)
> >
> > The reason for the slightly smaller pixel size is most probably reduced
> > floating point precision during creation of the CHELSA data.
>
> from personal comm.:
> They are currently investigating where the precision is cut in their
> workflow (the upcoming V1.2 shall come with a related correction).
>

Unfortunately they have not corrected the precision issue...

gdalinfo CHELSA_temp_2_1979-2013_V1.2_land.tif
Driver: GTiff/GeoTIFF
Files: CHELSA_temp_2_1979-2013_V1.2_land.tif
Size is 43200, 20880
[...]
Origin = (-180.00013850017,83.999860415150010)
Pixel Size = (0.0083330,-0.0083330)

Corner Coordinates:
Upper Left  (-180.0001389,  83.9998604) (180d 0' 0.50"W, 83d59'59.50"N)
Lower Left  (-180.0001389, -90.0001389) (180d 0' 0.50"W, 90d 0' 0.50"S)
Upper Right ( 179.9998597,  83.9998604) (179d59'59.49"E, 83d59'59.50"N)
Lower Right ( 179.9998597, -90.0001389) (179d59'59.49"E, 90d 0' 0.50"S)
Center  (  -0.0001396,  -3.0001392) (  0d 0' 0.50"W,  3d 0' 0.50"S)
Band 1 Block=43200x1 Type=Float32, ColorInterp=Gray
  NoData Value=-9

best,
Vero
___
grass-user mailing list
grass-user@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-user

[GRASS-user] New peer-review article about dynamic flood simulation with GRASS and Itzï

[Laurent C.]

Hello all,

A new open-access, peer-reviewed journal article has been published
about the use of GRASS and Itzï for the dynamic simulation of floods:

Courty, L. G., Pedrozo-Acuña, A., and Bates, P. D.: Itzï (version
17.1): an open-source, distributed GIS model for dynamic flood
simulation, Geosci. Model Dev., 10, 1835-1847,
doi:10.5194/gmd-10-1835-2017, 2017.

URL: http://www.geosci-model-dev.net/10/1835/2017/

I hope you find it interesting.

Best regards,
Laurent Courty
___
grass-user mailing list
grass-user@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] problems with d.mon windows

[Tyler Smith]

On Tue, May 2, 2017, at 12:05 PM, Tyler Smith wrote:
> I've been having some problems interacting with graphics windows. I'm
> using Grass version 7.2.0, installed from the Debian repository.
> Starting from the command line this happens:
> 
> grass -text
> 
> GRASS 7.2.0 (ecozones):~ > d.mon start=wx0
> GRASS 7.2.0 (ecozones):~ > Failed to import the site module
> Traceback (most recent call last):
>   File "/usr/lib/python3.5/site.py", line 580, in 
> main()

Realizing that these are python errors, I recalled that I have installed
a non-Debian (i.e., miniconda) version of Python that sits at the front
of my path. Adding `export GRASS_PYTHON=python2.7` to my .bashrc seems
to correct this issue. I have both the Debian python3.5 and the
miniconda python3.5, but using either as the value of GRASS_PYTHON
causes problems.

So I have solved my own problem, unless there will be new issues arising
from using python2.7 instead of python3+?

Thanks for your patience.

Tyler
___
grass-user mailing list
grass-user@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-user

[GRASS-user] problems with d.mon windows

[Tyler Smith]

Hi,

I've been having some problems interacting with graphics windows. I'm
using Grass version 7.2.0, installed from the Debian repository.
Starting from the command line this happens:

grass -text

GRASS 7.2.0 (ecozones):~ > d.mon start=wx0
GRASS 7.2.0 (ecozones):~ > Failed to import the site module
Traceback (most recent call last):
  File "/usr/lib/python3.5/site.py", line 580, in 
main()
  File "/usr/lib/python3.5/site.py", line 566, in main
known_paths = addusersitepackages(known_paths)
  File "/usr/lib/python3.5/site.py", line 287, in addusersitepackages
user_site = getusersitepackages()
  File "/usr/lib/python3.5/site.py", line 263, in getusersitepackages
user_base = getuserbase() # this will also set USER_BASE
  File "/usr/lib/python3.5/site.py", line 253, in getuserbase
USER_BASE = get_config_var('userbase')
  File "/usr/lib/python3.5/sysconfig.py", line 595, in get_config_var
return get_config_vars().get(name)
  File "/usr/lib/python3.5/sysconfig.py", line 538, in get_config_vars
_init_posix(_CONFIG_VARS)
  File "/usr/lib/python3.5/sysconfig.py", line 410, in _init_posix
from _sysconfigdata import build_time_vars
  File "/usr/lib/python3.5/_sysconfigdata.py", line 6, in 
from _sysconfigdata_m import *
ImportError: No module named '_sysconfigdata_m'

At this point, GRASS thinks wx0 is running:

GRASS 7.2.0 (ecozones):~ > d.mon -l
List of running monitors:
wx0

But I can't see it anywhere.

Yesterday I was able to get wx0 started, but I got strange and
inconsistent errors with the d.what.vect interface (i.e., clicking on
the icon in the window then clicking on a vector object in the window).
I can't reproduce that anymore as now I can't even get the monitor
started.

Have I broken something? How can I get my map windows back?

Thanks,

Tyler 

-- 
plantarum.ca
___
grass-user mailing list
grass-user@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] Error importing Mapset module

[Joseph Kariuki]

I have managed to group the raster bands following your command. And
removed the Mapset import as it is redundant for there is a Mapset defined
already. Thanks Helmut



Kind Regards,

*Joseph Kariuki*

*Geospatial Engineer** | **GIS / Web Developer*

On Tue, May 2, 2017 at 2:00 PM, Helmut Kudrnovsky  wrote:

> >ImportError: libgrass_datetime.7.2.1svn.so: cannot open shared >object
> file:
> No such file or directory
>
> => libgrass_datetime.7.2.1svn.so
>
> - Is GRASS compilation ok?
> - Does this file exist?
> - Can this file be found (e.g. in path)?
>
>
>
> -
> best regards
> Helmut
> --
> View this message in context: http://osgeo-org.1560.x6.
> nabble.com/Error-importing-Mapset-module-tp5318705p5319052.html
> Sent from the Grass - Users mailing list archive at Nabble.com.
> ___
> grass-user mailing list
> grass-user@lists.osgeo.org
> https://lists.osgeo.org/mailman/listinfo/grass-user
___
grass-user mailing list
grass-user@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] Error importing Mapset module

[Helmut Kudrnovsky]

>ImportError: libgrass_datetime.7.2.1svn.so: cannot open shared >object file:
No such file or directory

=> libgrass_datetime.7.2.1svn.so

- Is GRASS compilation ok?
- Does this file exist? 
- Can this file be found (e.g. in path)?



-
best regards
Helmut
--
View this message in context: 
http://osgeo-org.1560.x6.nabble.com/Error-importing-Mapset-module-tp5318705p5319052.html
Sent from the Grass - Users mailing list archive at Nabble.com.
___
grass-user mailing list
grass-user@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] Error importing Mapset module

[Joseph Kariuki]

This is the exact error I am getting from running the script

Traceback (most recent call last):
  File "/home/hempire/PycharmProjects/GRASSproject/landsatProcessing.py",
line 157, in 
from grass.pygrass.gis import Mapset
  File
"/usr/local/grass-7.2.1svn/etc/python/grass/pygrass/gis/__init__.py", line
13, in 
import grass.lib.gis as libgis
  File "/usr/local/grass-7.2.1svn/etc/python/grass/lib/gis.py", line 23, in

_libs["grass_gis.7.2.1svn"] = load_library("grass_gis.7.2.1svn")
  File "/usr/local/grass-7.2.1svn/etc/python/grass/lib/ctypes_loader.py",
line 62, in load_library
return self.load(path)
  File "/usr/local/grass-7.2.1svn/etc/python/grass/lib/ctypes_loader.py",
line 78, in load
raise ImportError(e)
ImportError: libgrass_datetime.7.2.1svn.so: cannot open shared object file:
No such file or directory




Kind Regards,

*Joseph Kariuki*

*Geospatial Engineer** | **GIS / Web Developer*

On Mon, May 1, 2017 at 12:30 PM, Markus Neteler  wrote:

> On Fri, Apr 28, 2017 at 10:30 AM, Joseph Kariuki 
> wrote:
> > I have shared a gist on pygrass landsat processing. Error importing
> Mapset
> > module which is under Pygrass>gis library. How can I import the module
> so as
> > to run i.group without error?
> >
> > The gist on github landsatprocessing_gist
> > https://gist.github.com/joehene/9225cff545776abdd1e4a934a3f6c11e
>
> You need to add  to use separator is "comma" to the g.list part in
> order to provide i.group with a comma separated list of maps.
>
> Example:
> i.group group=vis_bands subgroup=vis_bands
> input=lsat7_2000_10,lsat7_2000_20,lsat7_2000_30
>
> Markus
>
> --
> Markus Neteler, PhD
> http://www.mundialis.de - free data with free software
> http://grass.osgeo.org
> http://courses.neteler.org/blog
>
___
grass-user mailing list
grass-user@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-user