Re: [GRASS-user] r.sun.daily with multiple CPU cores: error uncompressing raster data ...

[Dylan Beaudette]

Update: after applying the latest patch, I now see

ERROR: Decompression failed with error -1

I found the map that fails decompression. Is there any way to inspect
the map in order to search for more clues as to what is wrong with it
or how it might have happened?


All of the maps in this project are using the default ZLIB
compression, along with compressed NULL files. Looking over the zlib
manual (https://www.zlib.net/manual.html), I see several references to
an error code of "-1":


#define Z_ERRNO(-1)

Z_ERRNO if there is an error writing the flushed data

Z_ERRNO on a file operation error

ZEXTERN const char * ZEXPORT gzerror OF((gzFile file, int *errnum));

Returns the error message for the last error which occurred on the
given compressed file. errnum is set to zlib error number. If an error
occurred in the file system and not in the compression library, errnum
is set to Z_ERRNO and the application may consult errno to get the
exact error code.


The last note is interesting and suggests that this specific "problem"
may be happening at the file system or OS level. That said, I have
only encountered this error in the context of FCELL or DCELL
maps--which makes me think that it is some combination of GRASS and
the underlying file system.

Some OS information:

3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6 15:44:32 UTC 2016 x86_64
x86_64 x86_64 GNU/Linux
Description:Ubuntu 14.04.5 LTS
Release:14.04


Thanks!
Dylan



On Sat, Dec 30, 2017 at 9:51 PM, Dylan Beaudette
<dylan.beaude...@gmail.com> wrote:
> Dang, this appears to be happening with any module run in parallel
> that generates FCELL or DCELL maps. In this case, r.horizon run in
> parallel.
>
> I have added additional commentary to #2764
>
> https://trac.osgeo.org/grass/ticket/2764#comment:9
>
> After application of Markus' latest patch the error message is now
> "Decompression failed with error 0".
>
> Dylan
>
> On Sat, Dec 30, 2017 at 9:34 PM, Dylan Beaudette
> <dylan.beaude...@gmail.com> wrote:
>> On Sat, Dec 30, 2017 at 2:32 PM, Markus Metz
>> <markus.metz.gisw...@gmail.com> wrote:
>>>
>>>
>>> On Fri, Dec 29, 2017 at 5:07 PM, Dylan Beaudette <dylan.beaude...@gmail.com>
>>> wrote:
>>>>
>>>> Hi everyone,
>>>>
>>>> First of all, thanks in advance for the r.sun.daily module which is a
>>>> nice replacement for my amateurish attempts over the last 12 years.
>>>>
>>>> I am currently working on an annual beam radiance map for a large
>>>> geographic region, at 30m res: 70,953 x 46,964 cells. This is far too
>>>> large for a single pass of r.horizon or r.sun on my machine so I have
>>>> split the data into 5,000 x 5,000 cell tiles with 100 cells of
>>>> overlap. This seems to be sufficient for my purposes and the edge
>>>> effects are tolerable.
>>>>
>>>> At 8-15 minutes / tile / day (r.sun) and 54 tiles this job calls for
>>>> multiple CPU cores. All of the parallel processing that I use is (as
>>>> far as I know) contained within the same region.
>>>>
>>>>
>>>> I have had good success with running r.horizon in parallel via GNU
>>>> parallel like this:
>>>>
>>>> # 1: start angle
>>>> # 2: angle step
>>>> # 3: elevation tile
>>>> seq 0 $step 330 | parallel --gnu --progress "bash make-hz-maps.sh {}
>>>> $step $elev"
>>>>
>>>> Which is just a wrapper around r.horizon and run in parallel "within"
>>>> tiles.
>>>>
>>>> Next, I run r.sun.daily (8 CPU cores) within tiles:
>>>>
>>>> r.sun.daily --overwrite elevation=$elev aspect=$asp slope=$slope \
>>>> start_day=1 end_day=365 beam_rad=$beam horizon_basename=hzangle
>>>> horizon_step=$step nprocs=8
>>>>
>>>>
>>>> The r.sun.daily modules finishes without error about 50-60% of the
>>>> time, results look good. The other 50-40% of the the time I get this:
>>>>
>>>>
>>>> ERROR: Error uncompressing raster data for row 2605 of
>>>>
>>>> *** buffer overflow detected ***: g.message terminated
>>>> === Backtrace: =
>>>> /lib/x86_64-linux-gnu/libc.so.6(+0x7329f)[0x7f8b9a70129f]
>>>> /lib/x86_64-linux-gnu/libc.so.6(__fortify_fail+0x5c)[0x7f8b9a79c83c]
>>>> /lib/x86_64-linux-gnu/libc.so.6(+0x10d710)[0x7f8b9a79b710]
>>>> /lib/x86_64-linux-gnu/libc.so.6(+0x10cc19)[0x7f8b9a79a

Re: [GRASS-user] r.sun.daily with multiple CPU cores: error uncompressing raster data ...

[Dylan Beaudette]

Dang, this appears to be happening with any module run in parallel
that generates FCELL or DCELL maps. In this case, r.horizon run in
parallel.

I have added additional commentary to #2764

https://trac.osgeo.org/grass/ticket/2764#comment:9

After application of Markus' latest patch the error message is now
"Decompression failed with error 0".

Dylan

On Sat, Dec 30, 2017 at 9:34 PM, Dylan Beaudette
<dylan.beaude...@gmail.com> wrote:
> On Sat, Dec 30, 2017 at 2:32 PM, Markus Metz
> <markus.metz.gisw...@gmail.com> wrote:
>>
>>
>> On Fri, Dec 29, 2017 at 5:07 PM, Dylan Beaudette <dylan.beaude...@gmail.com>
>> wrote:
>>>
>>> Hi everyone,
>>>
>>> First of all, thanks in advance for the r.sun.daily module which is a
>>> nice replacement for my amateurish attempts over the last 12 years.
>>>
>>> I am currently working on an annual beam radiance map for a large
>>> geographic region, at 30m res: 70,953 x 46,964 cells. This is far too
>>> large for a single pass of r.horizon or r.sun on my machine so I have
>>> split the data into 5,000 x 5,000 cell tiles with 100 cells of
>>> overlap. This seems to be sufficient for my purposes and the edge
>>> effects are tolerable.
>>>
>>> At 8-15 minutes / tile / day (r.sun) and 54 tiles this job calls for
>>> multiple CPU cores. All of the parallel processing that I use is (as
>>> far as I know) contained within the same region.
>>>
>>>
>>> I have had good success with running r.horizon in parallel via GNU
>>> parallel like this:
>>>
>>> # 1: start angle
>>> # 2: angle step
>>> # 3: elevation tile
>>> seq 0 $step 330 | parallel --gnu --progress "bash make-hz-maps.sh {}
>>> $step $elev"
>>>
>>> Which is just a wrapper around r.horizon and run in parallel "within"
>>> tiles.
>>>
>>> Next, I run r.sun.daily (8 CPU cores) within tiles:
>>>
>>> r.sun.daily --overwrite elevation=$elev aspect=$asp slope=$slope \
>>> start_day=1 end_day=365 beam_rad=$beam horizon_basename=hzangle
>>> horizon_step=$step nprocs=8
>>>
>>>
>>> The r.sun.daily modules finishes without error about 50-60% of the
>>> time, results look good. The other 50-40% of the the time I get this:
>>>
>>>
>>> ERROR: Error uncompressing raster data for row 2605 of
>>>
>>> *** buffer overflow detected ***: g.message terminated
>>> === Backtrace: =
>>> /lib/x86_64-linux-gnu/libc.so.6(+0x7329f)[0x7f8b9a70129f]
>>> /lib/x86_64-linux-gnu/libc.so.6(__fortify_fail+0x5c)[0x7f8b9a79c83c]
>>> /lib/x86_64-linux-gnu/libc.so.6(+0x10d710)[0x7f8b9a79b710]
>>> /lib/x86_64-linux-gnu/libc.so.6(+0x10cc19)[0x7f8b9a79ac19]
>>> /lib/x86_64-linux-gnu/libc.so.6(_IO_default_xsputn+0xbc)[0x7f8b9a70961c]
>>> /lib/x86_64-linux-gnu/libc.so.6(_IO_vfprintf+0x1cc5)[0x7f8b9a6d9905]
>>> /lib/x86_64-linux-gnu/libc.so.6(__vsprintf_chk+0x84)[0x7f8b9a79aca4]
>>>
>>> /usr/local/grass-7.5.svn/lib/libgrass_gis.7.5.svn.so(+0x1343c)[0x7f8b9aa6a43c]
>>>
>>> /usr/local/grass-7.5.svn/lib/libgrass_gis.7.5.svn.so(G_fatal_error+0xbf)[0x7f8b9aa6accf]
>>> g.message(main+0x254)[0x400dd4]
>>> /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf5)[0x7f8b9a6aff45]
>>> g.message[0x400ea2]
>>>
>>>
>>> I can't seem to replicate the problem, as subsequent runs with the
>>> same parameters and in the same tile are successful! This leads me to
>>> think that:
>>>
>>> * some aspect of this approach is not thread safe
>>> * there is something wrong with my computer
>>> * there is a subtle bug in the raster writing / reading code when
>>> invoked in parallel
>>>
>>>
>>> I have encountered similar raster reading errors in the past,
>>> typically in the context of parallel processing:
>>>
>>> https://trac.osgeo.org/grass/ticket/2762
>>>
>>> https://trac.osgeo.org/grass/ticket/2764
>>>
>>>
>>> http://osgeo-org.1560.x6.nabble.com/Error-reading-raster-data-for-row-xxx-only-when-using-r-series-and-t-rast-series-td5229569i20.html
>>>
>>> https://lists.osgeo.org/pipermail/grass-dev/2015-July/075691.html
>>>
>>>
>>> Here is some information on my system and version of GRASS:
>>>
>>>  ./configure  --enable-64bit --with-libs=/usr/lib --without-pthread
>>> --without-odbc --without-mysql --with-readline --with-cxx
>>

Re: [GRASS-user] r.sun.daily with multiple CPU cores: error uncompressing raster data ...

[Dylan Beaudette]

On Sat, Dec 30, 2017 at 2:32 PM, Markus Metz
<markus.metz.gisw...@gmail.com> wrote:
>
>
> On Fri, Dec 29, 2017 at 5:07 PM, Dylan Beaudette <dylan.beaude...@gmail.com>
> wrote:
>>
>> Hi everyone,
>>
>> First of all, thanks in advance for the r.sun.daily module which is a
>> nice replacement for my amateurish attempts over the last 12 years.
>>
>> I am currently working on an annual beam radiance map for a large
>> geographic region, at 30m res: 70,953 x 46,964 cells. This is far too
>> large for a single pass of r.horizon or r.sun on my machine so I have
>> split the data into 5,000 x 5,000 cell tiles with 100 cells of
>> overlap. This seems to be sufficient for my purposes and the edge
>> effects are tolerable.
>>
>> At 8-15 minutes / tile / day (r.sun) and 54 tiles this job calls for
>> multiple CPU cores. All of the parallel processing that I use is (as
>> far as I know) contained within the same region.
>>
>>
>> I have had good success with running r.horizon in parallel via GNU
>> parallel like this:
>>
>> # 1: start angle
>> # 2: angle step
>> # 3: elevation tile
>> seq 0 $step 330 | parallel --gnu --progress "bash make-hz-maps.sh {}
>> $step $elev"
>>
>> Which is just a wrapper around r.horizon and run in parallel "within"
>> tiles.
>>
>> Next, I run r.sun.daily (8 CPU cores) within tiles:
>>
>> r.sun.daily --overwrite elevation=$elev aspect=$asp slope=$slope \
>> start_day=1 end_day=365 beam_rad=$beam horizon_basename=hzangle
>> horizon_step=$step nprocs=8
>>
>>
>> The r.sun.daily modules finishes without error about 50-60% of the
>> time, results look good. The other 50-40% of the the time I get this:
>>
>>
>> ERROR: Error uncompressing raster data for row 2605 of
>>
>> *** buffer overflow detected ***: g.message terminated
>> === Backtrace: =
>> /lib/x86_64-linux-gnu/libc.so.6(+0x7329f)[0x7f8b9a70129f]
>> /lib/x86_64-linux-gnu/libc.so.6(__fortify_fail+0x5c)[0x7f8b9a79c83c]
>> /lib/x86_64-linux-gnu/libc.so.6(+0x10d710)[0x7f8b9a79b710]
>> /lib/x86_64-linux-gnu/libc.so.6(+0x10cc19)[0x7f8b9a79ac19]
>> /lib/x86_64-linux-gnu/libc.so.6(_IO_default_xsputn+0xbc)[0x7f8b9a70961c]
>> /lib/x86_64-linux-gnu/libc.so.6(_IO_vfprintf+0x1cc5)[0x7f8b9a6d9905]
>> /lib/x86_64-linux-gnu/libc.so.6(__vsprintf_chk+0x84)[0x7f8b9a79aca4]
>>
>> /usr/local/grass-7.5.svn/lib/libgrass_gis.7.5.svn.so(+0x1343c)[0x7f8b9aa6a43c]
>>
>> /usr/local/grass-7.5.svn/lib/libgrass_gis.7.5.svn.so(G_fatal_error+0xbf)[0x7f8b9aa6accf]
>> g.message(main+0x254)[0x400dd4]
>> /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf5)[0x7f8b9a6aff45]
>> g.message[0x400ea2]
>>
>>
>> I can't seem to replicate the problem, as subsequent runs with the
>> same parameters and in the same tile are successful! This leads me to
>> think that:
>>
>> * some aspect of this approach is not thread safe
>> * there is something wrong with my computer
>> * there is a subtle bug in the raster writing / reading code when
>> invoked in parallel
>>
>>
>> I have encountered similar raster reading errors in the past,
>> typically in the context of parallel processing:
>>
>> https://trac.osgeo.org/grass/ticket/2762
>>
>> https://trac.osgeo.org/grass/ticket/2764
>>
>>
>> http://osgeo-org.1560.x6.nabble.com/Error-reading-raster-data-for-row-xxx-only-when-using-r-series-and-t-rast-series-td5229569i20.html
>>
>> https://lists.osgeo.org/pipermail/grass-dev/2015-July/075691.html
>>
>>
>> Here is some information on my system and version of GRASS:
>>
>>  ./configure  --enable-64bit --with-libs=/usr/lib --without-pthread
>> --without-odbc --without-mysql --with-readline --with-cxx
>> --enable-largefile --with-freetype
>> --with-freetype-includes=/usr/include/freetype2 --with-sqlite
>> --with-python --with-geos=/usr/local/bin/geos-config --without-opencl
>> --with-opencl-includes=/usr/include/CL/ --with-postgres
>> --with-postgres-includes=/usr/include/postgresql/
>> --with-postgres-libs=/usr/lib/
>> --with-proj-share=/usr/local/share/proj/
>>
>> version=7.5.svn
>> date=2017
>> revision=r71964
>> build_date=2017-12-21
>> build_platform=x86_64-pc-linux-gnu
>> build_off_t_size=8
>>
>>
>> Any ideas?
>
> Please try the patch attached to ticket #2764 helps to get closer to the
> problem.
>
> Markus M
>

Hi Markus,

Thank you for the quick reply and patch. I have recompiled with the
patch from #2764 and now waiting to see what happens.

Happy New Year!

Dylan
___
grass-user mailing list
grass-user@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-user

[GRASS-user] r.sun.daily with multiple CPU cores: error uncompressing raster data ...

[Dylan Beaudette]

Hi everyone,

First of all, thanks in advance for the r.sun.daily module which is a
nice replacement for my amateurish attempts over the last 12 years.

I am currently working on an annual beam radiance map for a large
geographic region, at 30m res: 70,953 x 46,964 cells. This is far too
large for a single pass of r.horizon or r.sun on my machine so I have
split the data into 5,000 x 5,000 cell tiles with 100 cells of
overlap. This seems to be sufficient for my purposes and the edge
effects are tolerable.

At 8-15 minutes / tile / day (r.sun) and 54 tiles this job calls for
multiple CPU cores. All of the parallel processing that I use is (as
far as I know) contained within the same region.


I have had good success with running r.horizon in parallel via GNU
parallel like this:

# 1: start angle
# 2: angle step
# 3: elevation tile
seq 0 $step 330 | parallel --gnu --progress "bash make-hz-maps.sh {}
$step $elev"

Which is just a wrapper around r.horizon and run in parallel "within" tiles.

Next, I run r.sun.daily (8 CPU cores) within tiles:

r.sun.daily --overwrite elevation=$elev aspect=$asp slope=$slope \
start_day=1 end_day=365 beam_rad=$beam horizon_basename=hzangle
horizon_step=$step nprocs=8


The r.sun.daily modules finishes without error about 50-60% of the
time, results look good. The other 50-40% of the the time I get this:


ERROR: Error uncompressing raster data for row 2605 of
   
*** buffer overflow detected ***: g.message terminated
=== Backtrace: =
/lib/x86_64-linux-gnu/libc.so.6(+0x7329f)[0x7f8b9a70129f]
/lib/x86_64-linux-gnu/libc.so.6(__fortify_fail+0x5c)[0x7f8b9a79c83c]
/lib/x86_64-linux-gnu/libc.so.6(+0x10d710)[0x7f8b9a79b710]
/lib/x86_64-linux-gnu/libc.so.6(+0x10cc19)[0x7f8b9a79ac19]
/lib/x86_64-linux-gnu/libc.so.6(_IO_default_xsputn+0xbc)[0x7f8b9a70961c]
/lib/x86_64-linux-gnu/libc.so.6(_IO_vfprintf+0x1cc5)[0x7f8b9a6d9905]
/lib/x86_64-linux-gnu/libc.so.6(__vsprintf_chk+0x84)[0x7f8b9a79aca4]
/usr/local/grass-7.5.svn/lib/libgrass_gis.7.5.svn.so(+0x1343c)[0x7f8b9aa6a43c]
/usr/local/grass-7.5.svn/lib/libgrass_gis.7.5.svn.so(G_fatal_error+0xbf)[0x7f8b9aa6accf]
g.message(main+0x254)[0x400dd4]
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf5)[0x7f8b9a6aff45]
g.message[0x400ea2]


I can't seem to replicate the problem, as subsequent runs with the
same parameters and in the same tile are successful! This leads me to
think that:

* some aspect of this approach is not thread safe
* there is something wrong with my computer
* there is a subtle bug in the raster writing / reading code when
invoked in parallel


I have encountered similar raster reading errors in the past,
typically in the context of parallel processing:

https://trac.osgeo.org/grass/ticket/2762

https://trac.osgeo.org/grass/ticket/2764

http://osgeo-org.1560.x6.nabble.com/Error-reading-raster-data-for-row-xxx-only-when-using-r-series-and-t-rast-series-td5229569i20.html

https://lists.osgeo.org/pipermail/grass-dev/2015-July/075691.html


Here is some information on my system and version of GRASS:

 ./configure  --enable-64bit --with-libs=/usr/lib --without-pthread
--without-odbc --without-mysql --with-readline --with-cxx
--enable-largefile --with-freetype
--with-freetype-includes=/usr/include/freetype2 --with-sqlite
--with-python --with-geos=/usr/local/bin/geos-config --without-opencl
--with-opencl-includes=/usr/include/CL/ --with-postgres
--with-postgres-includes=/usr/include/postgresql/
--with-postgres-libs=/usr/lib/
--with-proj-share=/usr/local/share/proj/

version=7.5.svn
date=2017
revision=r71964
build_date=2017-12-21
build_platform=x86_64-pc-linux-gnu
build_off_t_size=8


Any ideas?

I haven't had a chance to inspect the maps in question as r.sun.daily
deletes the intermediate pieces on error.

Thanks!
Dylan
___
grass-user mailing list
grass-user@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] Parallel processes

[Dylan Beaudette]

On Mon, Oct 19, 2015 at 12:05 PM, Glynn Clements
<gl...@gclements.plus.com> wrote:
>
> Dylan Beaudette wrote:
>
>> Are there any reasons to prefer sequential operations (that do not
>> alter the region) vs. parallel operations?
>
> Running additional jobs in parallel is only worthwhile if the
> resources which they would use (CPU, memory, I/O bandwidth) would
> otherwise be idle.
>
> Once you get to the point that a resource is saturated and jobs are
> contending for it, parallel execution will be less efficient than
> serial execution.
>
> Maybe the "parallel" command takes these factors into account
> sufficiently. If it only considers CPU cores (i.e. one job per core),
> you'd need to confirm that you aren't saturating I/O bandwidth or
> thrashing memory or CPU caches. Try running the same sequence of tasks
> with varying numbers of parallel jobs to determine the optimal value.
> Needless to say, this will vary according to the nature of the task
> (e.g. I/O-bound versus CPU-bound).
>

Thank you Glynn, your advice confirms some empirical notes:

1. parallel processes that use data from external USB disks quickly
saturate the capacity of the bus or mechanism of the drive

2. parallel processes that use data from an internal SSD can generally
saturate all 8 cores of my Intel i7


My main motivation for asking this question was to determine instances
where parallel operations in GRASS are _not_ safe. From my reading of
the wiki, manual pages, and your recent comments on GRASS-dev, it
would appear that the following operations may not be safe:

1. region-altering

2. calculations in the presence of a MASK

3. reading "external" (r.external) GDAL sources (?)

4. some mapcalc expressions

In order to simplify my testing, I have disabled pthread support and
invoke "parallelization" via backgrounding or GNU parallel. My
examples with GNU parallel stem from the tremendous (apparent) utility
of this tool, in that most "bash for loops" can be directly converted
into "smart" parallel jobs.

Thanks,
Dylan
___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

[GRASS-user] Parallel processes

[Dylan Beaudette]

Hi,

Are there any reasons to prefer sequential operations (that do not
alter the region) vs. parallel operations?

For example:

# this
seq 1 30 | parallel -j8 --gnu --progress r.surf.gauss --o --q
output=testing_00{}

# vs.

# this
for map in `seq 1 30`
do
r.surf.gauss --o --q output=testing_00$map
done


I have consulted the relevant page on the wiki:

https://grasswiki.osgeo.org/wiki/Parallel_GRASS_jobs

... and it does appear to discourage the first example above. Anyone
else have some examples of this kind of workflow and potential
caveats?

I have noticed that there can be a significant speed bump when running
some tasks in parallel, especially when the source files are stored on
a SSD.

Thanks,
Dylan
___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] ERROR: Error reading raster data for row 210 of

[Dylan Beaudette]

Strange,

I used to get this type of error when running r.series on a large
number of maps, but seemingly at random. I would re-make the map that
"caused' the error, re-run r.series and it would usually work.

Time passed and I forgot about it. I assumed that updating to grass71
(trunk) was the solution.

This afternoon the error returned when using t.rast.series

Error reading raster data for row 292 of!!!

Again, these errors seem to happen at random. In this case, I am
iterating over a sequence of years and computing growing degree days
via t.rast.series. Here is the output from several years:

1984
WARNING: Overwriting space time raster dataset  and
 unregistering all maps
WARNING: Overwriting space time raster dataset  and
 unregistering all maps
Starting temporal sampling...
Starting mapcalc computation...
Starting map registration in temporal database...
WARNING: Overwriting space time raster dataset  and unregistering all
 maps
 100%
1985
WARNING: Overwriting space time raster dataset  and
 unregistering all maps
WARNING: Overwriting space time raster dataset  and
 unregistering all maps
Starting temporal sampling...
Starting mapcalc computation...
Starting map registration in temporal database...
WARNING: Overwriting space time raster dataset  and unregistering all
 maps
 100%
ERROR: Error reading raster data for row 292 of 
ERROR: r.series failed. Check above error messages.
1986
WARNING: Overwriting space time raster dataset  and
 unregistering all maps
WARNING: Overwriting space time raster dataset  and
 unregistering all maps
Starting temporal sampling...
Starting mapcalc computation...
Starting map registration in temporal database...
WARNING: Overwriting space time raster dataset  and unregistering all
 maps


... the stack of rasters called "gdd" is re-computed for each year.
Everything worked as expected in 1984, read error in 1985, fine again
in 1986. There is no MASK present.

I have only ever encountered these errors when using t.rast.series and
r.series. Does anyone have any ideas on how I can replicate or
otherwise debug these kind of errors? This is a serious problem for
long-running calculations.

Thanks!
Dylan




On Tue, Sep 8, 2015 at 3:49 AM, umberto.minora  wrote:
> Thank you, installing the newer version (7.0.1) solved the problem!
>
>
>
> --
> View this message in context: 
> http://osgeo-org.1560.x6.nabble.com/ERROR-Error-reading-raster-data-for-row-210-of-cumRAD-tp5222503p5223086.html
> Sent from the Grass - Users mailing list archive at Nabble.com.
> ___
> grass-user mailing list
> grass-user@lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/grass-user
___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] Splitting a location across several disks

[Dylan Beaudette]

Thanks, this is quite helpful. I'll report back with my strategy and
any limitations of this approach.

On Wed, Oct 7, 2015 at 11:43 PM, Blumentrath, Stefan
<stefan.blumentr...@nina.no> wrote:
> See also: https://grasswiki.osgeo.org/wiki/Location_and_Mapsets
> There you find a detailed description of how to link mapsets...
>
> -Original Message-
> From: grass-user-boun...@lists.osgeo.org 
> [mailto:grass-user-boun...@lists.osgeo.org] On Behalf Of Dylan Beaudette
> Sent: 8. oktober 2015 00:50
> To: Glynn Clements <gl...@gclements.plus.com>
> Cc: grass list <grass-user@lists.osgeo.org>
> Subject: Re: [GRASS-user] Splitting a location across several disks
>
> On Wed, Oct 7, 2015 at 3:39 PM, Glynn Clements <gl...@gclements.plus.com> 
> wrote:
>>
>>
>> Dylan Beaudette wrote:
>>
>> > It has been a while, but glad to be back on GRASS-user.
>> >
>> > I am working on a project that involves a significant storage
>> > dilemma: try and fit most of the files into a 500 Gb SSD for
>> > blazing-fast I/O, or fall back to a standard but higher capacity disk 
>> > drive.
>> >
>> > Would it be possible to store "derived" data into a mapset that is
>> > on standard disk, while the "source" data reside in another mapset,
>> > stored on the SSD?
>> >
>> > In other words, is it OK for a location to contain several mapsets
>> > that don't "live" on the same physical disk. It seems like it should
>> > work (via symlink), but I would like to see if there are any caveats
>> > that I should be aware of.
>>
>> If symlinks don't work, Linux supports "mount --bind ...", which lets
>> you mount a directory from an already-mounted filesystem at another
>> location. Windows has similar features (e.g. reparse points), although
>> I'm not that familiar with the specifics.
>>
>
> Excellent! This is the answer that I was looking for. I will try symlinks 
> first, otherwise the "mount --bind" strategy is simple enough. Fortunately 
> this work will be done on linux so just about anything is possible.
>
>>
>> The main constraint is that you can't split a single mapset across
>> devices, as it must be possible to rename() files in the .tmp
>> subdirectory to other directories withing the mapset, which requires
>> that they are on the same physical partition (rename() only
>> manipulates directory entries, it won't move the file's data blocks).
>
> OK, good to know.
>
> Is there any reason to think that reading lots of raster files will be 
> noticeably faster on the SSD?
>
> Thanks,
> Dylan
>
>> --
>> Glynn Clements <gl...@gclements.plus.com>
> ___
> grass-user mailing list
> grass-user@lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/grass-user
___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] External Database and projections

[Dylan Beaudette]

The use of r.external makes a lot of sense when dealing with very large
files. Does the use of "external" files work as expected in all of the new
t.* modules?

Thanks,
Dylan

On Wed, Oct 7, 2015 at 12:09 PM, Markus Neteler  wrote:

> On Wed, Oct 7, 2015 at 12:55 PM, Blumentrath, Stefan
>  wrote:
> ...
> > My suggestion is to not use PostGIS for big rasters, unless you have to,
> > because you want to use the data in a specific application for example.
>
> Note r.external and r.external.out of GRASS GIS 7 for avoiding data
> duplication:
>
> https://grass.osgeo.org/grass70/manuals/r.external.html
> https://grass.osgeo.org/grass70/manuals/r.external.out.html
>
> Here an example:
>
> # register (rather than import) a GeoTIFF file in GRASS GIS:
> r.external input=terra_lst1km20030314.LST_Day.tif output=modis_celsius
>
> # define output directory for files resulting from subsequent calculations:
> r.external.out directory=$HOME/gisoutput/ format="GTiff"
>
> # perform calculations (here: extract pixels > 20 deg C)
> # store output directly as GeoTIFF file, hence add the .tif extension:
> r.mapcalc "warm.tif = if(modis_celsius > 20.0, modis_celsius, null() )"
>
> # cease GDAL output connection and turn back to write standard GRASS
> raster files:
> r.external.out -r
>
> # use the result elsewhere
> qgis $HOME/gisoutput/warm.tif
>
> Best
> Markus
> ___
> grass-user mailing list
> grass-user@lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/grass-user
>
___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] Splitting a location across several disks

[Dylan Beaudette]

On Wed, Oct 7, 2015 at 3:39 PM, Glynn Clements <gl...@gclements.plus.com> wrote:
>
>
> Dylan Beaudette wrote:
>
> > It has been a while, but glad to be back on GRASS-user.
> >
> > I am working on a project that involves a significant storage dilemma: try
> > and fit most of the files into a 500 Gb SSD for blazing-fast I/O, or fall
> > back to a standard but higher capacity disk drive.
> >
> > Would it be possible to store "derived" data into a mapset that is on
> > standard disk, while the "source" data reside in another mapset, stored on
> > the SSD?
> >
> > In other words, is it OK for a location to contain several mapsets that
> > don't "live" on the same physical disk. It seems like it should work (via
> > symlink), but I would like to see if there are any caveats that I should be
> > aware of.
>
> If symlinks don't work, Linux supports "mount --bind ...", which lets
> you mount a directory from an already-mounted filesystem at another
> location. Windows has similar features (e.g. reparse points), although
> I'm not that familiar with the specifics.
>

Excellent! This is the answer that I was looking for. I will try
symlinks first, otherwise the "mount --bind" strategy is simple
enough. Fortunately this work will be done on linux so just about
anything is possible.

>
> The main constraint is that you can't split a single mapset across
> devices, as it must be possible to rename() files in the .tmp
> subdirectory to other directories withing the mapset, which requires
> that they are on the same physical partition (rename() only
> manipulates directory entries, it won't move the file's data blocks).

OK, good to know.

Is there any reason to think that reading lots of raster files will be
noticeably faster on the SSD?

Thanks,
Dylan

> --
> Glynn Clements <gl...@gclements.plus.com>
___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

[GRASS-user] Splitting a location across several disks

[Dylan Beaudette]

Hi,

It has been a while, but glad to be back on GRASS-user.

I am working on a project that involves a significant storage dilemma: try
and fit most of the files into a 500 Gb SSD for blazing-fast I/O, or fall
back to a standard but higher capacity disk drive.

Would it be possible to store "derived" data into a mapset that is on
standard disk, while the "source" data reside in another mapset, stored on
the SSD?

In other words, is it OK for a location to contain several mapsets that
don't "live" on the same physical disk. It seems like it should work (via
symlink), but I would like to see if there are any caveats that I should be
aware of.

Also, is the NULL file compression code ready for production use?

https://trac.osgeo.org/grass/ticket/2750


Thanks!
Dylan
___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] re-order boundaries by relative position along polyline

[Dylan Beaudette]

Thanks Thomas and Adrien! These are some great ideas. I'll be sure to
check out (haha) github repository when I have chance.

It turns out that a combination of GRASS + R did the trick. Essentially:

1. export snapshot of initial mapping linework via R
 * data are kept in their native, projected coordinate system
2. import into PostGIS
 * data are transformed to GCS WGS84
3. create UNION of exterior ring of initial mapping + interior
ring of adjacent polygons
 * a threshold of 0.0001 (~10 meters) is used to locate adjacent polygons
4. export to shapefile
 * data are converted back to native, projected coordinate system
5. import into GRASS
 * data are converted into a topological vector format with snapping
tolerance of 10 meters
6. generate join document in GRASS and export to shapefile
 * result is a set of line segments, one for each join decision
7. create a unique set of IDs for join decision line segments, based
on a hash of (rounded) segment start and end coordinates in R.

The solution-- which is still a work in progress-- is here:

https://github.com/dylanbeaudette/work-misc/tree/master/join-document

I'll be refining this considerably over the next two days for
field-testing the following week. Ideas welcome!

Dylan




On Tue, Apr 7, 2015 at 8:54 AM, Thomas Adams tea...@gmail.com wrote:
 Adrien,

 Thank you! I'll give it a try...

 Tom

 On Tue, Apr 7, 2015 at 10:44 AM, Adrien André adrien.an...@onf.fr wrote:

 Dylan, Tom,

 not about the main question, but to complete Tom's answer:

 I had once to vectorize a raster stream network according to flow
 direction.
 You can find here the Python script i wrote:
 https://github.com/adrienandrem/r.to.vect.dir

 I hope this helps.


 Regards,

 Adrien


 Le 05/04/2015 14:48, Thomas Adams a écrit :
  Hi Dylan!
 
  (I'm about to start a new position — more on that later). This may help…
 
  I had a problem somewhat analogous to this a little while ago, where I
  needed to create a text file that, among a couple of other things,
  needed to identify the 'downstream' pixel for all grid cells within a
  basin area. I was able to do that with help here, but an issue I had was
  that the integer IDs needed to be numbered from 0-N in basically
  upstream to downstream order, but after my GRASS analysis I had IDs with
  some large number ranging to some other very large number for the IDs
  and the 'downstream IDs' (with values skipped and not in order). I got
  help writing a Perl script that uses hashes for a lookup table to relate
  the old ID to the new ID.
 
  Perhaps you could do something like this, starting with a new vector map
  (copy) where you could simply update the attributes using an SQL query?
  Some coding would be needed, obviously…
 
  Just my 2 cents…
 
  Cheers!
  Tom
 
  On Thu, Apr 2, 2015 at 6:31 PM, Dylan Beaudette
  dylan.beaude...@gmail.com mailto:dylan.beaude...@gmail.com wrote:
 
  Hi,
 
  Once again, I am reminded that complex spatial analysis in other
  software
  can usually be reduced to a couple of lines of code in GRASS--
  thanks!
 
  Here is a fun question. Suppose you have a set of line segments, or
  boundaries that are the result of a digitizing process that
  generated
  segments out of order:
 
2 4  1   3
  O-+-++--O
 
  'O' = start / end point
  '+' = vertex
 
  ... in this case order is associated with order in which records
  appear
  in the attribute table and the order in which segments are drawn on
  the
  screen.
 
  Is there a module, or set of modules that could be used to re-order
  the
  segments so that the order of records in the attribute table and
  drawing
  order is:
 
1 2  3   4
  O-+-++--O
 
  Note that I do not wan't to shuffle the linkages between geometry
  and
  attributes, rather, I would like to re-order the segments and
  attribute
  table according to the linear position within the set of contiguous
  boundaries or polyline.
 
  Here is an example set of lines in ASCII format, WGS84 GCS.
 
  ORGANIZATION:
  DIGIT DATE:
  DIGIT NAME:   dylan
  MAP NAME:
  MAP DATE: Thu Apr  2 11:46:34 2015
  MAP SCALE:1
  OTHER INFO:
  ZONE: 0
  MAP THRESH:   0.00
  VERTI:
  B  6 1
   -118.71960108 36.71846736
   -118.71946261 36.71783146
   -118.71905508 36.71717314
   -118.71863837 36.71610377
   -118.71807413 36.71546352
   -118.71786286 36.71527706
   2 132
  B  3 1
   -118.7153795 36.73403608
   -118.71537992 36.73272737
   -118.71538933 36.73177277
   2 133
  B  3 1
   -118.71536851 36.76205202
   -118.7153724 36.7609144
   -118.71537619 36.75980591
   2 150
  B  2 1
   -118.73128842 36.69749903

[GRASS-user] re-order boundaries by relative position along polyline

[Dylan Beaudette]

Hi,

Once again, I am reminded that complex spatial analysis in other software
can usually be reduced to a couple of lines of code in GRASS-- thanks!

Here is a fun question. Suppose you have a set of line segments, or
boundaries that are the result of a digitizing process that generated
segments out of order:

  2 4  1   3
O-+-++--O

'O' = start / end point
'+' = vertex

... in this case order is associated with order in which records appear
in the attribute table and the order in which segments are drawn on the
screen.

Is there a module, or set of modules that could be used to re-order the
segments so that the order of records in the attribute table and drawing
order is:

  1 2  3   4
O-+-++--O

Note that I do not wan't to shuffle the linkages between geometry and
attributes, rather, I would like to re-order the segments and attribute
table according to the linear position within the set of contiguous
boundaries or polyline.

Here is an example set of lines in ASCII format, WGS84 GCS.

ORGANIZATION:
DIGIT DATE:
DIGIT NAME:   dylan
MAP NAME:
MAP DATE: Thu Apr  2 11:46:34 2015
MAP SCALE:1
OTHER INFO:
ZONE: 0
MAP THRESH:   0.00
VERTI:
B  6 1
 -118.71960108 36.71846736
 -118.71946261 36.71783146
 -118.71905508 36.71717314
 -118.71863837 36.71610377
 -118.71807413 36.71546352
 -118.71786286 36.71527706
 2 132
B  3 1
 -118.7153795 36.73403608
 -118.71537992 36.73272737
 -118.71538933 36.73177277
 2 133
B  3 1
 -118.71536851 36.76205202
 -118.7153724 36.7609144
 -118.71537619 36.75980591
 2 150
B  2 1
 -118.73128842 36.69749903
 -118.73130166 36.69760244
 2 175
B  5 1
 -118.71538933 36.73177277
 -118.7153944 36.73125742
 -118.71573607 36.73044156
 -118.71618977 36.72875872
 -118.71651025 36.72811846
 2 581
B  15 1
 -118.71651025 36.72811846
 -118.71661716 36.72766138
 -118.71660977 36.7273294
 -118.71645785 36.72668322
 -118.71689758 36.7261392
 -118.71724087 36.72567564
 -118.71747107 36.7248177
 -118.71753527 36.72390024
 -118.71787045 36.72307308
 -118.71797278 36.72237579
 -118.71781832 36.72073331
 -118.71825483 36.72004703
 -118.71908689 36.71948153
 -118.71963768 36.71863545
 -118.71960108 36.71846736
 2 602
B  8 1
 -118.7180943 36.70872005
 -118.71827421 36.7083356
 -118.71841915 36.7080259
 -118.7190169 36.70752707
 -118.72008403 36.70694237
 -118.72042476 36.70636815
 -118.72049695 36.70608245
 -118.7212077 36.70547966
 2 604
B  15 1
 -118.7212077 36.70547966
 -118.721692  36.70506892
 -118.72256889 36.70442599
 -118.72372477 36.702873
 -118.72417965 36.70213899
 -118.72496002 36.70190631
 -118.72660809 36.70183505
 -118.72765407 36.70118732
 -118.72831767 36.70100376
 -118.72866574 36.7007615
 -118.72887421 36.70026501
 -118.72998965 36.69849669
 -118.73066541 36.69798083
 -118.73118943 36.69770441
 -118.73130166 36.69760244
 2 609
B  8 1
 -118.7153867 36.75138736
 -118.7153804 36.7500365
 -118.7153634 36.7464107
 -118.7153465 36.742785
 -118.7153296 36.7391593
 -118.71533791 36.73753527
 -118.7153438 36.7363838
 -118.7153795 36.73403608
 2 610
B  13 1
 -118.71786286 36.71527706
 -118.71763335 36.71507451
 -118.71748992 36.71480774
 -118.71754381 36.7136
 -118.71750723 36.7131568
 -118.71786592 36.71228789
 -118.71745106 36.71129755
 -118.71716489 36.71079564
 -118.71709787 36.71043286
 -118.71712906 36.71006868
 -118.71740871 36.70978478
 -118.717644  36.709369
 -118.7180943 36.70872005
 2 612
B  7 1
 -118.71411256 36.77582538
 -118.71445897 36.77550631
 -118.71478471 36.77508229
 -118.7149828 36.77446623
 -118.71532537 36.77406797
 -118.7153339 36.7717931
 -118.71534777 36.76808916
 2 613
B  3 1
 -118.71534777 36.76808916
 -118.71536   36.7645405
 -118.71536851 36.76205202
 2 622
B  4 1
 -118.71537619 36.75980591
 -118.7153848 36.7572883
 -118.7153973 36.7536622
 -118.7153867 36.75138736
 2 623


I have been able to convert these segments into polylines, however, I am
not sure how one would go about computing the position along the polylines.
Perhaps a linear referencing system?



Thanks!
Dylan
___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] Re: spgrass6

[Dylan Beaudette]

On Wednesday, February 22, 2012, Paolo Cavallini wrote:
 Il 22/02/2012 15:54, Roger Bivand ha scritto:
  Using the improved raster graphics handling for square cells with image()
  rather than spplot() and useRaster=TRUE - equivalent to
  image.SpatialGridDataFrame() and useRasterImage=TRUE with the same matrix
  takes 1.2 seconds on x11/cairo. You didn't say which version of R you are
  using - the raster graphics facilities have been improved recently.
 2.14.1-1 on Debian testing.
  Did you try using image() instead of spplot() if your cells are square, 
and
  if rasterImage() is available in your version of R?
 Yes, image() is resonably fast, but misses automatic legend, etc.
 Thanks a lot for your reply.
 
 -- 
 Paolo Cavallini
 See: http://www.faunalia.it/pc
 
 ___
 grass-user mailing list
 grass-user@lists.osgeo.org
 http://lists.osgeo.org/mailman/listinfo/grass-user
 

Check out the plot() function from the raster package, or the related 
rasterViz (?) package. They have some neat stuff in there. 

Roger already mentioned this, but I'll add a little more. Now that we have 
powerful computers and nice packages like spgrass6 it is very simple to load 
large raster files into R without much effort. However, just because we can 
fit a linear model on that massive stack of pixels doesn't mean that one 
should. This is even more important with regard to plotting functions. I have 
found that coarsening the region resolution a bit before loading in raster 
that only need to be plotted helps considerably. 

If you really must work with stacks of large grids, the best approach I have 
found is the raster package.

Cheers,
Dylan


-- 
Dylan E. Beaudette
USDA-NRCS Soil Scientist
California Soil Resource Lab
http://casoilresource.lawr.ucdavis.edu/
___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] GRASS GIS development visualization from 1999 to 2011

[Dylan Beaudette]

On Friday, November 11, 2011, Markus Neteler wrote:
 Hi,
 
 I have created a video of the GRASS GIS development from 1999 to 2011:
 
 http://grass.osgeo.org/wiki/GRASS_development_visualization
 
 Enjoy
 Markus

Really cool. I wonder if it would be possible to view the GRASS 6 -- GRASS 7 
branching + backporting efforts.

Dylan

-- 
Dylan E. Beaudette
USDA-NRCS Soil Scientist
California Soil Resource Lab
http://casoilresource.lawr.ucdavis.edu/
___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] resampling question

[Dylan Beaudette]

On Thursday, October 20, 2011, Kirk Wythers wrote:
 I am curious how the r.resamp.stats function (when used to aggregate with 
the method=mode switch) handles an equal number of different cell scores? 
For example, lets say that aggregating 16 cells to 1cell yields 8 cells with a 
value of 4, and 8 cells with a value of 5. In this case what would 
r.resamp.stats report as the mode? And what method would it use to determine 
that value?

The neat thing about GRASS and other OSS, you can always take a peak at the 
source code. From my basic understanding of C, I don't think that there is 
anything here that would accommodate ties-- so the result of 
mode(4,4,4,4,5,5,5,5) would probably be 4 -- as it comes first. This is the 
behavior in R when using something like which.max(table(x)).

Here is what I found in: lib/stats/c_mode.c
--
void c_mode(DCELL * result, DCELL * values, int n, const void *closure)
{
DCELL mode;
int max;
DCELL prev;
int count;
int i;

n = sort_cell(values, n);

max = 0;
count = 0;

for (i = 0; i  n; i++) {
if (max == 0 || values[i] != prev) {
prev = values[i];
count = 0;
}

count++;

if (count  max) {
max = count;
mode = prev;
}
}

if (max == 0)
Rast_set_d_null_value(result, 1);
else
*result = mode;
}
--


-- 
Dylan E. Beaudette
USDA-NRCS Soil Scientist
California Soil Resource Lab
http://casoilresource.lawr.ucdavis.edu/
___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] resampling question

[Dylan Beaudette]

On Thursday, October 20, 2011, Kirk Wythers wrote:
 On Oct 20, 2011, at 9:22 AM, Dylan Beaudette wrote:
 
  The neat thing about GRASS and other OSS, you can always take a peak at 
the 
  source code. From my basic understanding of C, I don't think that there is 
  anything here that would accommodate ties-- so the result of 
  mode(4,4,4,4,5,5,5,5) would probably be 4 -- as it comes first. This is 
the 
  behavior in R when using something like which.max(table(x)).
 
 Thanks for the reply Dylan. As I read through the code you pasted, it looks 
to me that as the variable 'values' gets counted, then in the case of a tie, 
such as this 2 x 2 matrix 
 
 _
 | 4 | 5 |
 _
 | 4 | 5 |
 _
 
 would yield a mode result of 4 with the logic that the tie goes to the 
smallest value in sorted list of values? In this case...  4 4 5 5 = mode of 4, 
where as a sorted list such as 1 2 3 4 would yield a mode of 1? 
 
 Is that right? 
 

That is my interpretation of the code. Perhaps Glynn or someone else more 
familiar with the raster library can comment further. Also- why not try a 
simple experiment. You can create fake raster data with a text editor and read 
it in via r.in.ascii.

put the following in a text file called 'fake.rast' :
--
north:  4
south:  0
east:   4
west:   0
rows:   4
cols:   4
4 4 4 4
4 4 4 4
5 5 5 5
5 5 5 5


# import
r.in.ascii in=fake.raster out=f
g.region rast=f -p

# check
d.mon wx0
d.rast.num f

# aggregate
g.region res=4 -p
r.resamp.stats --o in=f out=f.mode method=mode

# check
d.erase
d.rast.num f.mode


... which shows that our interpretation is correct. An interesting side-note:

# region is still set to a resolution of 4x4 units = 1 cell
# NN-resampling of the original data gives us 5...
r.out.ascii --q -h f
5 

# aggregated (mode) gives us 4...
r.out.ascii --q -h f.mode
4

Fun!


-- 
Dylan E. Beaudette
USDA-NRCS Soil Scientist
California Soil Resource Lab
http://casoilresource.lawr.ucdavis.edu/
___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] voronoi tesselation for vector/raster patches

[Dylan Beaudette]

On Tuesday, July 19, 2011, Milton Cezar Ribeiro wrote:
 Dear all,
 
 Can I generate voronoi tesselation for polygons (instead of points) of
 raster-based patches?
 
 cheers
 
 milton
 

What exactly would the process you describe result in, i.e. do you have any 
examples of voronoi tesselations derived from polygons? If the polygons are 
small-ish (relative to the exent of your analysis), then voronoi tesselations 
of the polygon centroids might be useful. Someone might have a better answer 
if the problem is more completely described, and even better if you provide a 
workable example from one of the sample data sets (e.g. spearfish).


-- 
Dylan E. Beaudette
USDA-NRCS Soil Scientist
California Soil Resource Lab
http://casoilresource.lawr.ucdavis.edu/
___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] gradient (slope) of a road

[Dylan Beaudette]

On Tue, May 31, 2011 at 8:54 AM, Markus Neteler nete...@osgeo.org wrote:
 On Tue, May 31, 2011 at 9:22 AM, Francisco Calzada
 paco.calzada2...@gmail.com wrote:
 Dear list,
 I'm new to this list and I wonder if you could help me.

 Welcome!

 I need to calculate the gradient (slope) of a road. I have a fine DEM to do
 it. The problem is when I run r.slope.aspect the resulting slope map have
 extremelly and odd high values. I don't know why. Do you? Anyway, could you
 tell me how do you calculate the slope of a road?

 First use v.drape to drape the 2D road map over the DEM:
 http://grass.osgeo.org/grass64/manuals/html64_user/v.drape.html

 Then use the resulting 3D vector map to calculate the slope for
 the vector segments using the slope method, results go into
 the attribute table:
 http://grass.osgeo.org/grass64/manuals/html64_user/v.to.db.html

 Hope this helps
 Markus

Markus' suggestion is the simplest. If you need more than slope,
consider densifying your line segments and sampling the raster of
interest along the new, shorter, segments. Here is the outline:

http://casoilresource.lawr.ucdavis.edu/drupal/node/698

Cheers,
Dylan
___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] Possible to model exatmospheric radiation with r.sun?

[Dylan Beaudette]

On Sunday, May 22, 2011, Hamish wrote:
 Jonathan Greenberg wrote:
  I was wondering if there is a way to turn the
  atmosphere off in r.sun, and model a top of
  atmosphere radiation?
 
 I wonder if it enough to keep the cloud coefficients (i.e. coefbh, coefdh
 r.rescale'd GRIB files) at the default clear-sky conditions, and set
 the Linke coefficient at lin=1.00.
 
 then compare how well the direct beam result matches the solar constant?
 (aka the see what happens experimental approach)
 
 
 Hamish
 
 

I would be interested in the result, and happy to test. Any tips on getting 
started with the coef maps and GRIB (not sure what that is)?

Dylan

-- 
Dylan E. Beaudette
USDA-NRCS Soil Scientist
California Soil Resource Lab
http://casoilresource.lawr.ucdavis.edu/
___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] Possible to model exatmospheric radiation with r.sun?

[Dylan Beaudette]

On Monday, May 16, 2011, Jonathan Greenberg wrote:
 Folks:
 
 I was wondering if there is a way to turn the atmosphere off in
 r.sun, and model a top of atmosphere radiation?
 
 --j

Hi Jonathan,

If you check out the metadata associated with the output from r.sun, you will 
find the following: (note the Extraterrestrial irradiance entry)

r.info rad_beam.187
 ++
 | Layer:rad_beam.187   Date: Thu Mar 24 17:17:43 2011
|
 | Mapset:   PERMANENT  Login of Creator: dylan   
|
 | Location: ca630
|
 | DataBase: /home/dylan/grass
|
 | Title: ( rad_beam.187 )
|
 | Timestamp: none
|
 ||
 |
|
 |   Type of Map:  raster   Number of Categories: 255 
|
 |   Data Type:FCELL  
|
 |   Rows: 1396   
|
 |   Columns:  553
|
 |   Total Cells:  771988 
|
 |Projection: UTM (zone 10)   
|
 |N:4235950S:4221990   Res:10 
|
 |E: 697230W: 691700   Res:10 
|
 |   Range of data:min = 4492.042  max = 7758.583 
|
 |
|
 |   Data Description:
|
 |generated by r.sun  
|
 |
|
 |   Comments:
|
 |    
|
 | Day [1-365]:  187  
|
 | Solar constant (W/m^2):   1367 
|
 | Extraterrestrial irradiance (W/m^2):  1321.303439  
|
 | Declination (rad):0.396193 
|
 | Latitude min-max(deg):38.1250 - 38.2500
|
 | Sunrise time min-max (hr.):   4.72 - 4.72  
|
 | Sunset time min-max (hr.):19.28 - 19.28
|
 | Time step (hr.):  0.5000   
|
 | Linke turbidity factor:   3.0  
|
 | Ground albedo:0.200
|
 | -  
|
 |
|
 |r.sun -s elevin=ned10m aspin=aspect10m aspect=270 slopein=slope\   
|
 |10m slope=0.0 lin=3.0 alb=0.2 beam_rad=rad_beam.187 day=187 step=\   
|
 |0.5 dist=1.0 numpartitions=1
|



 -- 
 Jonathan A. Greenberg, PhD
 Assistant Project Scientist
 Center for Spatial Technologies and Remote Sensing (CSTARS)
 Department of Land, Air and Water Resources
 University of California, Davis
 One Shields Avenue
 Davis, CA 95616
 Phone: 415-763-5476
 AIM: jgrn307, MSN: jgrn...@hotmail.com, Gchat: jgrn307
 ___
 grass-user mailing list
 grass-user@lists.osgeo.org
 http://lists.osgeo.org/mailman/listinfo/grass-user
 


-- 
Dylan E. Beaudette
USDA-NRCS Soil Scientist
California Soil Resource Lab
http://casoilresource.lawr.ucdavis.edu/
___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] r.resamp.rst dynamic tension?

[Dylan Beaudette]

Did you folks make any progress on this matter? I am interested in the 
addition of variable tension to the RST suite of tools. Already we have had 
great success with the variable smoothness option.

Cheers,
Dylan

On Wednesday, March 30, 2011, Hamish wrote:
 Hi,
 
 I am working with some u,v component wind data on a ~30 km grid which I
 wish to resample to a 3 km grid before r.what extractions at various
 fixed points. Often the winds have smooth transitions, so the small
 smoothing towards the mean that the spline does is not really a problem.
 Results so far with overlap=5 and tension=10 are very very nice.
 Stunning actually.. most of the time.
 
 One thing I am worried about though is that it will smooth out the peaks,
 which can be very important and who's effects are non-linear - e.g. in
 fronts or the eye-wall of a hurricane. Conserving the area under the
 curve is not enough, it must preserve the peaks as well (due to dramatic
 effects of velocity^2, etc.).
 
 the r.resamp.rst module has a smooth=raster_map option, I thought to
 maybe take the r.slope.aspect slope (1st deriv) of the overall magnitude
 map and apply it as dynamic smoothing, so calm areas allow a relaxed fit
 while areas of great change pull tighter. I guess that's more dynamic
 tension than dynamic smoothing though.
 
 the manual says:
 
For noisy data it is possible to define spatially variable
 smoothing by providing  a raster  map  named  by  the smooth option
 containing smoothing parameters.  With the smoothing parameter set to
 zero (smooth is not given or  contains  zero  data),  the resulting
 surface passes exactly through the data points.
 
 
 which sounds like it should already try to pass through outliers exactly
 as long as I don't use a smooth= map? As mentioned above, currently I'm
 using tension=10 and overlap=5 to get rid of outliers, probably
 introduced from an earlier resampling..
 
 
 a comparison of r.univar mean and std.dev. shows a slight reduction in
 overall range in the RST output map, but still closer to the original
 than a r.resamp.interp bilinear or bicubic interpolation.
 
 
 Losing the localized peaks due to grid aliasing is another concern, but
 there's less I can do about that AFAICT.
 
 
 any ideas?
 
 
 thanks,
 Hamish
 ___
 grass-user mailing list
 grass-user@lists.osgeo.org
 http://lists.osgeo.org/mailman/listinfo/grass-user
 


-- 
Dylan E. Beaudette
USDA-NRCS Soil Scientist
California Soil Resource Lab
http://casoilresource.lawr.ucdavis.edu/
___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

[GRASS-user] Re: [GRASS-dev] d.barb: call for testing

[Dylan Beaudette]

On Wednesday, April 27, 2011, Hamish wrote:
 Dylan wrote:
  Nice work! 
 
 cheers
  
  I had been thinking about using this type of graphic to
  describe the potential movement of water/sediment flux along
  landscape gradients for some time now... 
  With a little bit of tinkering around, I was able to get a
  figure that matched my mental picture:
  
  http://basho.dyndns.org/~dylan/temp/d.barb_terrain_example.jpg
  
  Colors represent mean curvature, contours are every 10
  meters, white lines are from r.flow, arrow direction is from
  an aspect map, arrow length is scaled by the compound
  topographic index.
 
 for the record, d.rast.arrow has been able to do this for some
 time now, see the GRASS raster screenshots page for an example.
 (actually for the d.barb Eulerian array fields I just adapted
 my earlier scaled arrow code from there)
 
 
 a tip to get rid of the outer cell-border of garbage values:
 r.mapcalc invert = if(isnull(map), 1, null()
 r.grow in=invert out=invert_buf1
 r.mapcalc MASK = if(isnull(invert_buf1), 1, null()
 r.mapcalc crop = map
 
 
 Hamish
 
 

Thanks for the tips, that worked well.

Dylan


-- 
Dylan E. Beaudette
USDA-NRCS Soil Scientist
California Soil Resource Lab
http://casoilresource.lawr.ucdavis.edu/
___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

[GRASS-user] Re: [GRASS-dev] d.barb: call for testing

[Dylan Beaudette]

Nice work! 

I had been thinking about using this type of graphic to describe the potential 
movement of water/sediment flux along landscape gradients for some time now... 
With a little bit of tinkering around, I was able to get a figure that matched 
my mental picture:

http://basho.dyndns.org/~dylan/temp/d.barb_terrain_example.jpg

Colors represent mean curvature, contours are every 10 meters, white lines are 
from r.flow, arrow direction is from an aspect map, arrow length is scaled by 
the compound topographic index.

Thanks!

Dylan


On Wednesday, April 27, 2011, Hamish wrote:
 Hi,
 
 I have added a new C module in the addons svn which will draw wind barbs,
 straw plots, and arrow plots from a raster array or sparse vector point
 data. It can use either direction + magnitude, or u + v components as the
 input, and can produce a legend key.
 
 Some of this functionality is already covered by d.rast.arrow and d.vect,
 but I thought I'd bring it together into a dedicated tool and add some
 commonly needed options.
 
 It's still a work in progress, and there are still a few things on the
 todo list, but I think it's ready to get some testing now.
 
 Once the bugs are out and the design  feature set have been finalized
 I'd port it to grass7 and hope to add it to the main distribution + GUI.
 (One weird thing I notice is that in the tcl/tk GUI the legend_at=
 option appears as a tick box not a text entry field. ?)
 
 
   http://grass.osgeo.org/wiki/GRASS_AddOns#d.barb
   http://trac.osgeo.org/grass/browser/grass-addons/display/d.barb
 
 screenshot:  (style=arrow)
   http://bambi.otago.ac.nz/hamish/grass/screenshots/narr-
a_221_20100629_1800_000_10m_winds.png
 
 
 maybe g.extension works to install it if you compiled grass yourself?
  (I haven't tried installing it that way yet)
 
 comments, wishes,  criticisms welcome.  usage follows, see the help
 page for more explanation  examples.
 
 
 Hamish
 
 
 
 --
 
 GRASS65 d.barb --help
 
 Description:
  Draws flow barbs.
 
 Keywords:
  
 
 Usage:
  d.barb [-r] [direction=name] [magnitude=name] [u=name] [v=name]
[input=name] [layer=value] [style=string] [color=name] [skip=value]
[scale=value] [peak=value] [aspect_type=string]
[legend_at=x,y[,x,y,...]] [legend_velo=value[,value,...]]
[legend_fontsize=value] [--verbose] [--quiet]
 
 Flags:
   -r   Rotate direction 180 degrees
 Useful for switching between atmospheric and oceanographic 
conventions
  --v   Verbose module output
  --q   Quiet module output
 
 Parameters:
 direction   Raster map (or attribute column) containing velocity 
direction
 magnitude   Raster map (or attribute column) containing velocity 
magnitude
 u   Raster map (or attribute column) containing u-component 
of velocity
 v   Raster map (or attribute column) containing v-component 
of velocity
 input   Name of input vector map
 layer   Layer number
  A single vector map can be connected to multiple 
database tables. This number determines which table to use.
 default: 1
 style   Style
 options: arrow,barb,straw
 default: arrow
 color   Color
  Either a standard color name or R:G:B triplet
 default: black
  skip   Draw arrow every Nth grid cell
 default: 10
 scale   Scale factor for arrow rendering
 default: 1.0
  peak   Maximum value for scaling (overrides map's maximum)
   aspect_type   Direction map aspect type
 options: cartesian,compass
 default: cartesian
 legend_at   Screen percentage for legend barb ([0,0] is bottom-left)
  Draws a single barb and exits
 options: 0-100
 default: 10.0,10.0
   legend_velo   Velocity for legend key arrow
   legend_fontsize   Font size used in legend
 default: 14
 ___
 grass-dev mailing list
 grass-...@lists.osgeo.org
 http://lists.osgeo.org/mailman/listinfo/grass-dev
 


-- 
Dylan E. Beaudette
USDA-NRCS Soil Scientist
California Soil Resource Lab
http://casoilresource.lawr.ucdavis.edu/
___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

[GRASS-user] grass on an Android Phone

[Dylan Beaudette]

Hi,

Has anyone heard of a successful installation of GRASS (and associated 
toolchain) on an Android phone? They run linux, but probably lack on-board 
resources for hosting an entire development toolkit. However... it seems like 
it would be possible to use a cross-compiler to generate a statically-linked 
set of binaries (or pile of required .so files) for this platform...

Some ideas here:
http://android-dls.com/wiki/index.php?title=Compiling_for_Android
http://www.kbrandt.com/2009/06/how-to-cross-compile-the-bash-shell-for-
android-15.html
http://android-tricks.blogspot.com/2009/02/hello-world-c-program-on-using-
android.html

It would be pretty cool to get GRASS, or something like GRASS running on 
Android devices.

Cheers,
Dylan
___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] hotspot or heat map in GRASS

[Dylan Beaudette]

On Friday, February 04, 2011, razmjoo...@faunalia.co.uk wrote:
 Dear all
 
 Does anyone have a tutorial or an example of how to create heatmaps or
 hotspots based on point density in GRASS.
 
 I tried to use v.kernel but not sure what some of the parameters mean?
 v.neighbor is straight forward but the output map is very coarse.
 


Hi,

v.kernel is the way to go, however its output will depend on a couple things:

1. region settings
2. the size of the kernel (in terms of standard deviation)
3. obviously, the point density

tinker around with those, starting with the region settings (e.g. the 
resolution), then tinker with the standard deviation (in map units) option.

Cheers,
Dylan
___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] SVN for grass projects

[Dylan Beaudette]

grass add-ons site?

Dylan

On Wed, Aug 18, 2010 at 2:45 PM, Jonathan Greenberg
greenb...@ucdavis.edu wrote:
 Grass'ers:

 I've recently learned the wonders of SVN, and was hoping to drop this
 r.gridengine code I've been working on (it parallizes r.* commands
 using grid engine) someplace so I can use SVN, and also have the code
 easily available if anyone wants to try it out.  I can use
 sourceforge, but I didn't know if there was a more grass-friendly host
 location that would be more appropriate (like
 http://r-forge.r-project.org/ is for R packages).

 Thanks!

 --j
 ___
 grass-user mailing list
 grass-user@lists.osgeo.org
 http://lists.osgeo.org/mailman/listinfo/grass-user

___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] 1-page grass overview

[Dylan Beaudette]

On Wednesday 04 August 2010, Hamish wrote:
 Hi,

 for the upcoming FOSS4G conference[1] we have put together a live
 dvd[2] with GRASS and a whole bunch of other OSGeo software,
 about 40 FOSS geo projects are participating in all.

 It would be great if folks could read through the 1-page overview
 doc[3] for GRASS (made with reStructured Text and Sphinx if you
 wondered), and the 5 minute quickstart tutorial[4].

 A full list of project docs on the disc is interesting to look
 through as well.[5]

 Also if anyone is really keen they can download an ISO of the
 DVD[6] and test that it actually works. I've already discovered
 that the new QGIS 1.5 grass plugin doesn't like the user not
 owning the PERMANENT mapset, even if it's not the one being used.

 Test results to the OSGeo wiki[7] and bugs to the osgeo bug
 trac'er[8] please (including doc bugs; set component as LiveDVD).


 it goes to the printers next week.


 [1] http://2010.foss4g.org

 [2] http://live.osgeo.org

 [3]
 http://download.osgeo.org/livedvd/4.0-rc6/osgeolive-docs/overview/grass_ove
rview.html

 [4]
 http://download.osgeo.org/livedvd/4.0-rc6/osgeolive-docs/quickstart/grass_q
uickstart.html

 [5]
 http://download.osgeo.org/livedvd/4.0-rc6/osgeolive-docs/overview/overview.
html

 [6] http://download.osgeo.org/livedvd/4.0-rc6/

 [7] http://wiki.osgeo.org/wiki/Live_GIS_Disc_Testing

 [8] https://trac.osgeo.org/osgeo/report/10



 thanks,
 Hamish




Just wanted to mention that the product looks very polished and the short 
summary of GRASS is well done. Great work!

Dylan



-- 
Dylan Beaudette
Soil Resource Laboratory
http://casoilresource.lawr.ucdavis.edu/
University of California at Davis
530.754.7341
___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

[GRASS-user] Distance-weighted focal functions?

[Dylan Beaudette]

Hi,

After hacking together some R + r.mapcalc code, I started to wonder about how 
hard it would be to add distance-weighting to r.neighbors... Specifically, 
the focal distance-weighted mean difference. An example of what I was trying 
to accomplish can be found here:

http://casoilresource.lawr.ucdavis.edu/drupal/node/937

Surely there is a more elegant solution using r.mfilter.fp, or the like... 

Cheers,
Dylan

-- 
Dylan Beaudette
Soil Resource Laboratory
http://casoilresource.lawr.ucdavis.edu/
University of California at Davis
530.754.7341
___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] Cluster whit area?

[Dylan Beaudette]

 On 07/27/2010 01:30 PM, Etienne DELAY (deust 2010) wrote:
  I'm come back :-)
  I want to try clustTool in GRASS,
  so I reset the region setting ,
  I run R in grass,
  I run library(clustTool)
  That give mi a graphical environement, but I can't choose the data where
  I want to work!!! I just can choose exemple data...
 
  may be a basic question how to work on my data?
  sorry for this noob question...:- S


Some ideas here:
http://casoilresource.lawr.ucdavis.edu/drupal/node/340

Dylan



-- 
Dylan Beaudette
Soil Resource Laboratory
http://casoilresource.lawr.ucdavis.edu/
University of California at Davis
530.754.7341
___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] elevation profile

[Dylan Beaudette]

This might be useful:

http://casoilresource.lawr.ucdavis.edu/drupal/node/375

Cheers,
Dylan

On Thursday 15 July 2010, John Callahan wrote:
 Hopefully a quick question that I'm struggling with.  (I'm a long-time GIS
 user but trying to move my work to GRASS and other FOSS4G.)   I'm using
 GRASS 6.4 RC6 on Windows XP.

 I have a raster grid representing elevation at 2meter resolution. I have a
 line shapefile representing roads (about 20 roads total).  I would like to
 get ASCII text files of x,y,z at about 1 or 2 meter intervals along each
 road.  Any ides of what I can try?

 I thought v.drape might work. This converts my 2D road shapefile to 3D, but
 I'm not sure where to go from there.  And this doesn't help getting 1 or 2
 meter intervals.  I also tried r.profile but this requires input as a
 listing of x,y coordinates, not shapefiles.  I know there's a step here I'm
 missing.

 - John

 **
 John Callahan, Geographic Information Scientist
 Delaware Geological Survey, University of Delaware
 URL: http://www.dgs.udel.edu
 **



-- 
Dylan Beaudette
Soil Resource Laboratory
http://casoilresource.lawr.ucdavis.edu/
University of California at Davis
530.754.7341
___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] min/max of a set of rasters?

[Dylan Beaudette]

On Friday 25 June 2010, Jonathan Greenberg wrote:
 What is the quickest way to calculate, at a given g.region, the
 min/max of a set of rasters (which may have different resolutions) for
 each pixel?  Is this something that mapcalc can do?

 --j
 ___
 grass-user mailing list
 grass-user@lists.osgeo.org
 http://lists.osgeo.org/mailman/listinfo/grass-user

Hi Jonathan,

check out

r.univar
r.series

Dylan

-- 
Dylan Beaudette
Soil Resource Laboratory
http://casoilresource.lawr.ucdavis.edu/
University of California at Davis
530.754.7341
___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] Temporal Analysis

[Dylan Beaudette]

On Monday 07 June 2010, Nikos Dumakis wrote:
 Thank you Martin.

 I realized that one of the functions that I need, (counter of values bellow
 a certain threshold) is not available in r.series. Do you know, or do you
 have any suggestion, how to perform this?
 Thanks

 Best regards
 Nikos

Hi Nikos,

I can't seem to find it right now, but there was a discussion a while back 
regarding user-defined functions for r.series. I don't think anything came 
out of it, but one of the Martins or Markus' (I think) was able to put some 
nice working into a more consolidated stats library. Others may recall more 
details.

The basic premise: implement a mechanism by which a vector (i.e. the array of 
values defined by a single cell but sliced through a time series of rasters) 
could be passed to an R function, python function, etc. that would do its 
work and return a scalar to GRASS. This seems like it would be hard to 
implement, but would add tremendous flexibility to time series analysis. 

Another approach might be to use the 'raster' package for R. 

Cheers,
Dylan


 On Mon, Jun 7, 2010 at 1:12 PM, Martin Wegmann 

 martin.wegm...@uni-wuerzburg.de wrote:
  Hello Nikos,
 
  just give r.series a try.
 
  regards, Martin
 
  On Montag, 7. Juni 2010 14:06:17 Nikos Dumakis wrote:
   Greetings
  
  
   I have to do some simple-statistical TEMPORAL analysis between a long
   data-series. I mean, I have do some temporal average for each pixel,
   minimum and max. Instead of doing this statistical- analysis for all
   pixels in an image, I have to do this analysis for each image-pixel
   along its temporal series.
  
   Is there any function to do this? Or as anyone tried ?
   Thanks
   Nikos
 
  --
  Dr. Martin Wegmann
 
  University of Wuerzburg
  Institute of Geography
  Department of Remote Sensing
  Remote Sensing and Ecosystem Research
  Am Hubland
  97074 Wuerzburg, Germany
  @
  German Aerospace Center (DLR)
  German Remote Sensing Data Center (DFD)
 
  Phone:  +49-(0)931-31-83446
  Fax:   +49-(0)931-31-84961
  Email:  martin.wegm...@uni-wuerzburg.de
  url: http://www.remote-sensing.uni-wuerzburg.de
  url: http://www.dlr.de/caf
  url: http://www.global-change-ecology.de



-- 
Dylan Beaudette
Soil Resource Laboratory
http://casoilresource.lawr.ucdavis.edu/
University of California at Davis
530.754.7341
___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] calculating road density

[Dylan Beaudette]

Hi,

A couple options:

1. convert lines to points and use v.kernel or v.neighbors

2. import lines into R, and use the density function from the spatstat package.

Then, sample the density rasters within your polygon of interest with
v.rast.stats, etc.

Cheers,
Dylan

On Wed, May 26, 2010 at 1:30 AM, maning sambale
emmanuel.samb...@gmail.com wrote:
 Hi,

 I have two vectors in GRASS:
 road network - line
 grid - polygon

 I want to compute road density and append the attributes to the grid poly.
 In QGIS I tried using the vector tools (sum line length) with
 shapefiles and it works for small areas bu not for my full region.
 Any ideas?

 --
 cheers,
 maning
 --
 Freedom is still the most radical idea of all -N.Branden
 wiki: http://esambale.wikispaces.com/
 blog: http://epsg4253.wordpress.com/
 --
 ___
 grass-user mailing list
 grass-user@lists.osgeo.org
 http://lists.osgeo.org/mailman/listinfo/grass-user

___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] Latitude/Longitude vs UTM

[Dylan Beaudette]

On Fri, May 14, 2010 at 6:22 AM, Rich Shepard rshep...@appl-ecosys.com wrote:
 On Thu, 13 May 2010, Kurt Springs wrote:

 This was interesting in that it told me that r.topidx could not be run
 with latitude and longitude and I had to convert to UTM. I was wondering
 if this is the answer to the problem and I just had to convert to UTM.

 Kurt,

  Lat/Long represents geographic coordinates, not a projection of location
 on a mathematial model of the earth. UTM is the Universal Transverse
 Mercador projection that we see on most printed (or computer displayed) maps
 of the earth. There is documentation within the GRASS Web site that provides
 a good explanation of the differences. GRASS modules work on geographic
 projections, not just coordinates.

  There is a USGS technical report from the mid-1980s that's the standard on
 projections. While it is becoming more rare to locatate, see if you can find
 a copy.

I think that Rich is referring to this USGS document, #1395

http://pubs.er.usgs.gov/usgspubs/pp/pp1395

Definitely worth the price if you want to become an expert in map projections.


 One other question. New Hampshire appears to fall within two UTM zones
 (19T and 20T) Is there a way for a maps set to contain two UTM zones?

Yes. Don't use UTM. In this case use a regional projection that suits
your needs:

1) navigation -- use a conformal projection
2) area statistics -- use an equal-area projection
... etc ...

Variations on the Albers or Lambert (conformal) conic projections work
quite well for large regions that are wider than tall, but for such as
small state should be just fine. We use an Albers equal-area
projection to house soil survey data for the lower 48 states.

  Interesting. NH is a tall, narrow state so one would assume it would be
 within a single zone. Regardless, yes there is a way to reproject locations
 in one zone on the other, but it's non trivial and I've not done it.

I wouldn't recommend it. The desirable properties of the UTM system
(i.e. the fairly good compromise between distortion, preservation of
angles, and preservation of area) only occur within a zone's
boundaries. The farther you move from the central meridian of the UTM
zone, the more distortion you will encounter-- therefore 'projecting'
UTM z10 data into UTM z11 is technically possible, but not a great
idea.

  Oregon is primarily in Zone 10, but the eastern edge (I don't recall the
 distance within the state) is in Zone 11. The available DEM and hydrologic
 data were reprojected from 11 to 10 by the supplying agency.

Hmm...

Dylan

 Rich
 ___
 grass-user mailing list
 grass-user@lists.osgeo.org
 http://lists.osgeo.org/mailman/listinfo/grass-user

___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] Entering '1' and 'BDL' in Database

[Dylan Beaudette]

On Monday 19 April 2010, Rich Shepard wrote:
For a scientific database holding observed and measured values there are
 two non-numeric (NAN) legitimate entries: 1 and 'below detection limits.'
 Is anyone aware of a standard way of expressing these?

The problem is that the column needs to be of a numeric type because the
 numbers will be manipulated mathematically and summarized statistically. So
 a NAN such as '1' or 'BDL' doesn't cut it.

Any thoughts?


how about NULL ?

Dylan


-- 
Dylan Beaudette
Soil Resource Laboratory
http://casoilresource.lawr.ucdavis.edu/
University of California at Davis
530.754.7341
___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] Entering '1' and 'BDL' in Database

[Dylan Beaudette]

On Monday 19 April 2010, Rich Shepard wrote:
 On Mon, 19 Apr 2010, Dylan Beaudette wrote:
  how about NULL ?

 Dylan,

NULL means unknown, and 'less than 1' or 'below detection limits' means
 it cannot be measured with the instruments used. There's a big difference.
 NULL can result from the measurement not being taken or the analysis not
 being conducted.

 Thanks,

 Rich

Hi,

In that case, how about another column that contains quality control 
information. There isn't really any mechanism in most relational database 
models that would allow you to store 'mixed' datatypes AND interpret them as 
a numeric datatype... to the best of my knowledge. 

Cheers,
Dylan



-- 
Dylan Beaudette
Soil Resource Laboratory
http://casoilresource.lawr.ucdavis.edu/
University of California at Davis
530.754.7341
___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] Stratified sampling in GRASS

[Dylan Beaudette]

On Wednesday 14 April 2010, joel dinis wrote:
 Hello to all,

 I'm trying to do a stratified
 sampling in GRASS, and I found
 nothing in GRASS about it.

 Does anyone knows how to do it?

 Thank in advance
 JD

Here is one approach with R:

http://casoilresource.lawr.ucdavis.edu/drupal/node/644

This example can be modified to read directly from a GRASS database with the 
spgrass6 package for R.

Cheers,
Dylan

-- 
Dylan Beaudette
Soil Resource Laboratory
http://casoilresource.lawr.ucdavis.edu/
University of California at Davis
530.754.7341
___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] Random points within a polygon?

[Dylan Beaudette]

On Thursday 18 March 2010, Hamish wrote:
 Gary wrote:
  I want to use v.random but I noticed it puts random points
  within the region, can I limit this to a polygon?

 Yes, see the v.random.cover addon script from wiki addons.

 Also there are many options available from the R statistics sp
 library. (see the R page in the grass wiki, once loaded do
 library(sp); demo(gallery).

 And of course the spgrass6 R-GRASS interface to transfer data.


Here is an example of exactly what Hamish suggested:

http://casoilresource.lawr.ucdavis.edu/drupal/node/644

Note that you can substitute readOGR() with readVECT6() to read in a grass 
vector file.

Cheers,
Dylan



-- 
Dylan Beaudette
Soil Resource Laboratory
http://casoilresource.lawr.ucdavis.edu/
University of California at Davis
530.754.7341
___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] Display of Terrain Perameter Maps

[Dylan Beaudette]

On Monday 01 March 2010, Rich Shepard wrote:
The output maps from r.slope.aspect. r.resamp.rst, and even r.topidx the
 output maps have values for each cell rather than trends. This is very
 difficult to interpret by non-technical viewers.

I've seen maps of slopes, profile and planar curvature, and topographic
 wetness indices that are more smoothly colored. If one wants to determine
 the curvature of slopes adjacent to a specific cell at its base it would be
 easier to see this if the values do not change with each cell. When we
 stand at the foot of a hill we tend to not focus on each detail of the
 terrain but the overall slope, profile curvature, and planar curvature.
 It's this type of overview that I'm trying to display.

Is there a way to get smoother output maps that are visually more
 informative to non-specialists?

 Rich
 ___
 grass-user mailing list
 grass-user@lists.osgeo.org
 http://lists.osgeo.org/mailman/listinfo/grass-user

Hi Rich,

I have found that with a certain amount of smoothing (r.param.scale , 
r.neighbors) and the right color scheme (r.colors [-e]) one can generate 
pretty nice looking maps that display overall trends. 

One thing to remember-- most GRASS modules are geared towards analysis-- 
making pretty maps comes second.

Dylan

-- 
Dylan Beaudette
Soil Resource Laboratory
http://casoilresource.lawr.ucdavis.edu/
University of California at Davis
530.754.7341
___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] Display of Terrain Perameter Maps

[Dylan Beaudette]

On Tuesday 02 March 2010, Rich Shepard wrote:
 On Tue, 2 Mar 2010, Dylan Beaudette wrote:
  I have found that with a certain amount of smoothing (r.param.scale ,
  r.neighbors) and the right color scheme (r.colors [-e]) one can generate
  pretty nice looking maps that display overall trends.

 Dylan,

I used r.param.scale extensively; will learn r.neighbors.

  One thing to remember-- most GRASS modules are geared towards analysis--
  making pretty maps comes second.

That's very true. However, when the analytical results need to be
 presented to non-technical decision-makers (lawyers, regulators,
 politicians) it's vital that the presentations be visually attractive. It
 adds credibility to the analytical results and, like typeset docuements
 rather than word processed ones, makes them more easily accepted.


I agree 100% -- All I was getting at was that the nice map-making requires 
some creativity. If you have some time check out GMT-- it can generate some 
very nice looking maps in PS/EPS format. There are some scripts on the GRASS 
add-ons wiki page with some ideas. 

Cheers,
Dylan



-- 
Dylan Beaudette
Soil Resource Laboratory
http://casoilresource.lawr.ucdavis.edu/
University of California at Davis
530.754.7341
___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] Calculating Length Between Points On Stream Network

[Dylan Beaudette]

Hi Richard,

Try the v.net.* modules.

Cheers,
Dylan

On Sat, Feb 27, 2010 at 11:09 AM, Rich Shepard rshep...@appl-ecosys.com wrote:
  I have a vector map of a stream network and want to find the distance
 between two points on the main stem. I have the geographic coordinates of
 each point on the line representing the stream. I tried v.what but I'm not
 comfortable that gave me the correct answer.

  The line between the two points is multiple segments where there are
 confluences with feeder streams.

  What module, used with the GUI or on the command line, will allow me to
 calculate the distance along the stream between two identified points?

 Rich
 ___
 grass-user mailing list
 grass-user@lists.osgeo.org
 http://lists.osgeo.org/mailman/listinfo/grass-user

___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

[GRASS-user] strange output from v.voronoi

[Dylan Beaudette]

Hi,

This morning I noticed very odd output from v.voronoi, specifically on split 
between adjacent points was missing and the output messages contained the 
following:

Number of nodes: 22
Number of primitives: 27
Number of points: 0
Number of lines: 0
Number of boundaries: 20
Number of centroids: 7
Number of areas: 6
Number of isles: 1
Number of duplicate centroids: 1

The resulting voronoi polygons are attached.

Here are some details on the vector:

Number of points:   127

type   countminmax
point127  1127
line   0  0  0
boundary   0  0  0
centroid   0  0  0
area   0  0  0
face   0  0  0
kernel 0  0  0
all  127  1127

Here is the version information:
GRASS 6.5.svn (2010) 

 ./configure  --with-tcltk-includes=/usr/include/tcl8.4 --with-postgres 
--without-odbc --with-mysql --with-mysql-includes=/usr/include/mysql/ 
--with-freetype --with-freetype-includes=/usr/include/freetype2 --with-readline 
--with-cxx --enable-largefile 
--with-postgres-includes=/usr/local/pgsql/include/ 
--with-postgres-libs=/usr/local/pgsql/lib/ --with-sqlite --with-python 
--with-proj-share=/usr/local/share/proj/ --with-cairo
Revision: 38990 
Date: 2009-09-05 10:01:13 -0700 (Sat, 05 Sep 2009) 


Could it be that the bad output is caused by some fundamental problem with the 
input point vector?

Cheers,
Dylan


-- 
Dylan Beaudette
Soil Resource Laboratory
http://casoilresource.lawr.ucdavis.edu/
University of California at Davis
530.754.7341
attachment: bad_thiessen.png___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] Measuring Stream Lengths

[Dylan Beaudette]

On Friday 29 January 2010, Rich Shepard wrote:
I want to measure the stream length between two identified points on a
 vector map.

v.distance doesn't seem to be the module to use.

Perhaps v.net.path? I don't know that the table for the streams map has
 all the required information.

Advice?

 Thanks,

 Rich


Hi Rich,

Check out v.to.db for uploading things like line segment length to the 
attribute table of your vectorized stream lines.

Dylan


-- 
Dylan Beaudette
Soil Resource Laboratory
http://casoilresource.lawr.ucdavis.edu/
University of California at Davis
530.754.7341
___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] r.param.scale Incorrect Output

[Dylan Beaudette]

On Thursday 28 January 2010, Rich Shepard wrote:
 On Thu, 28 Jan 2010, Rich Shepard wrote:
   I'm stymied trying to get output from r.param.scale that is equivalent
  to the example on the bottom of the man page.

Resetting g.region to the input elevation map without any additional
 options and adding a zmult factor of 5 appears to help. When the specified
 param is 'features' I see what should be ridges and channels, but no peaks
 or pits.

Changing the calculated param to either elev or profc yields blank maps
 when I try to display them with d.rast.

Suggestions on how to properly apply this module will be very helpful.

 Rich
 ___

Hi,

Are you working with data in a projected coordinate system? Are you x,y and z 
coordinates all in the same units. What is the general nature of the terrain 
you are looking at? Try adjusting the window size to get generalized 
features-- remember that the quantification of terrain is a fractal problem.

Cheers,
Dylan


-- 
Dylan Beaudette
Soil Resource Laboratory
http://casoilresource.lawr.ucdavis.edu/
University of California at Davis
530.754.7341
___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] Searching Docs about 3D geological modelisation

[Dylan Beaudette]

On Saturday 09 January 2010, Benjamin Ducke wrote:
 Cheers for these. They are certainly all highly interesting.
 Do you have an actual link for the T-PROGS software itself?
 All I can seem to come up with are interfaces from other
 software and publications mentioning it.

 I would certainly be interested in taking a look at your
 GRASS interface. Is T-PROGS open source?

 My gut feeling is that the T-PROGS approach would give better
 results than 3D kriging, as it seems better able to to
 follow 3D shape trends:

 http://chl.erdc.usace.army.mil/chl.aspx?p=sa=ARTICLES;37g=50

 ... but that certainly would need testing.

 Having said that, I also like this approach for a more
 heuristic model:

 http://chl.erdc.usace.army.mil/chl.aspx?p=sa=ARTICLES;41g=50

 It's very simple and could easily be implemented directly
 in GRASS GIS. In fact, I coded something very similar to this
 for archaeological stratigraphy reconstruction a while back.

 Cheers,

 Ben

Hi Ben and others,

Here are some concerns from the author of the TPROGS software:

-
Steve is hesitant because he's not sure what the finished product would be. I 
think he's probably concerned about misapplication or perhaps some kind of 
ripoff. Can you provide a bit more background on where you see this going?
-

I think that it would be helpful to put together a small proposal, regarding 
how the TPROGS source code / ideas would be integrated into GRASS. It seems 
like the author is worried about use without citation, and once he 
understands what GRASS developers have in mind, should be for it. 

To start the discussion, I propose that the methods used within the TPROGS 
software be integrated (with proper citations) into a GRASS library, so that 
a series of modules can perform the multi-step process associated with 
modeling transition probabilities. Furthermore, the GRASS rast3 (voxel) 
datatype should be used to store the resulting structures-- this will make 
visualization with NVIZ / Paraview a snap.

Alternatively, we may be able to link GRASS with TPROGS with a little bit of 
python glue. While this may work if there are limitations regarding the use 
of the TPROGS source, I think that having these algorithms present in the 
GRASS libraries would be a real benefit.

I have CC-ed Graham, so that  we can keep him in the conversation.

Cheers,
Dylan



 - Original Message -
 From: Dylan Beaudette dylan.beaude...@gmail.com
 To: Benjamin Ducke benjamin.du...@oxfordarch.co.uk
 Cc: GRASS user list grass-user@lists.osgeo.org
 Sent: Saturday, January 9, 2010 4:30:40 AM GMT +01:00 Amsterdam / Berlin /
 Bern / Rome / Stockholm / Vienna Subject: Re: [GRASS-user] Searching Docs
 about 3D geological modelisation

 Two more ideas:

 1. conditional simulation, based on a 3D variogram model
 2. transition probability-based interpolation of categories

 Check out gstat for the conditional simulation, and TPROGS for the
 transition probability. If anything is interested, I have done some
 programming to connect GRASS and TPROGS.

 Cheers!

 Dylan

 On Fri, Jan 8, 2010 at 1:24 PM, Benjamin Ducke

 benjamin.du...@oxfordarch.co.uk wrote:
  Woohoo, this forum is always a treasure trove
  of good advice. I had not idea SGemS existed!
  The Voronoi idea is also good, I am just not sure
  that the 3D Voronoi diagram is quite what one
  would instinctively think it is.
 
  http://en.wikipedia.org/wiki/Voronoi_diagram
 
  says: In general a cross section of a 3D Voronoi
  tessellation is not a 2D Voronoi tessellation itself.
 
  Need to look into that.
 
  I don't have much practical experience
  with Bayes models, so can't really comment on
  that.
 
  Cheers,
 
  Ben
 
  Christian Kaiser wrote:
  It seems to me that this is a 3D interpolation problem with categorical
  variables.
 
  Maybe the Bayesian Maximum Entropy approach could help. There are some
  interesting publications around also for geology and soil sciences, and
  they can deal with categorical data as well. Look for example here:
  http://www.enge.ucl.ac.be/staff/curr/Bogaert/biblioBME/BMEbibsubject.htm
 l#Soil%20Science
 
  Or maybe you can have a look at SGeMS (http://sgems.sourceforge.net), a
  tool for 3D geostatistics.
 
  None of them is available through GRASS, but the algorithms are freely
  available (I think open-source, but not verified).
 
  I am not a geologist, so please forgive if it is not adequate...
 
  Christian Kaiser
 
  On 8 janv. 2010, at 11:04, Benjamin Ducke wrote:
  Rich Shepard wrote:
  material. There is no interpolation algorithm in GRASS currently
  which can
  handle that sort of data well.
 
   So what is needed is a political algorithm. :-)
 
  That's actually right: given the presence of n different
  layer types in the vicinity of an empty voxel

Re: [GRASS-user] Searching Docs about 3D geological modelisation

[Dylan Beaudette]

Quick update:

I recently heard back from Graham Fogg here on campus, and he is in favor of 
allowing T-PROGS to be used within GRASS. However, he is still waiting for 
the final go-ahead from the original author.

Dylan

On Monday 11 January 2010, Thomas Adams wrote:
 Dylan,

 Can you tell me how to obtain TPROGS? Is it only available commercially?

 Thanks,
 Tom

 Dylan Beaudette wrote:
  Two more ideas:
 
  1. conditional simulation, based on a 3D variogram model
  2. transition probability-based interpolation of categories
 
  Check out gstat for the conditional simulation, and TPROGS for the
  transition probability. If anything is interested, I have done some
  programming to connect GRASS and TPROGS.
 
  Cheers!
 
  Dylan
 
  On Fri, Jan 8, 2010 at 1:24 PM, Benjamin Ducke
 
  benjamin.du...@oxfordarch.co.uk wrote:
  Woohoo, this forum is always a treasure trove
  of good advice. I had not idea SGemS existed!
  The Voronoi idea is also good, I am just not sure
  that the 3D Voronoi diagram is quite what one
  would instinctively think it is.
 
  http://en.wikipedia.org/wiki/Voronoi_diagram
 
  says: In general a cross section of a 3D Voronoi
  tessellation is not a 2D Voronoi tessellation itself.
 
  Need to look into that.
 
  I don't have much practical experience
  with Bayes models, so can't really comment on
  that.
 
  Cheers,
 
  Ben
 
  Christian Kaiser wrote:
  It seems to me that this is a 3D interpolation problem with categorical
  variables.
 
  Maybe the Bayesian Maximum Entropy approach could help. There are some
  interesting publications around also for geology and soil sciences, and
  they can deal with categorical data as well. Look for example here:
  http://www.enge.ucl.ac.be/staff/curr/Bogaert/biblioBME/BMEbibsubject.ht
 ml#Soil%20Science
 
  Or maybe you can have a look at SGeMS (http://sgems.sourceforge.net), a
  tool for 3D geostatistics.
 
  None of them is available through GRASS, but the algorithms are freely
  available (I think open-source, but not verified).
 
  I am not a geologist, so please forgive if it is not adequate...
 
  Christian Kaiser
 
  On 8 janv. 2010, at 11:04, Benjamin Ducke wrote:
  Rich Shepard wrote:
  material. There is no interpolation algorithm in GRASS currently
  which can
  handle that sort of data well.
 
   So what is needed is a political algorithm. :-)
 
  That's actually right: given the presence of n different
  layer types in the vicinity of an empty voxel, the algorithm
  would need to decide by some sort of majority vote
  which type to assign to that voxel.
 
   Kidding aside, I suspect that a fuzzy interpolation algorithm would
  solve the problem.
 
  How? You could make the interpolated value depend on a
  fuzzy set member function, I suppose, but the situation
  here is actually so well defined that I think a probabilistic
  approach would be preferable. Since each voxel can only
  store one value, a second output map could store the
  classification probability. That may be very useful
  for visualization (you could show voxels with little
  probability hazier).
 
  Ben
 
  Rich
  ___
  grass-user mailing list
  grass-user@lists.osgeo.org
  http://lists.osgeo.org/mailman/listinfo/grass-user
 
  --
  Benjamin Ducke
  Geospatial Consultant
 
  Oxford Archaeology Digital
  Janus House
  Osney Mead
  OX2 0ES
  Oxford, U.K.
 
  Tel: +44 (0)1865 263 800 (switchboard)
  Tel: +44 (0)1865 980 758 (direct)
  Fax :+44 (0)1865 793 496
  benjamin.du...@oadigital.net
  http://oadigital.net
 
 
 
 
 
  --
  Files attached to this email may be in ISO 26300 format (OASIS Open
  Document Format). If you have difficulty opening them, please visit
  http://iso26300.info for more information.
 
  ___
  grass-user mailing list
  grass-user@lists.osgeo.org
  http://lists.osgeo.org/mailman/listinfo/grass-user
 
  ___
  grass-user mailing list
  grass-user@lists.osgeo.org
  http://lists.osgeo.org/mailman/listinfo/grass-user
 
  --
  Benjamin Ducke
  Geospatial Consultant
 
  Oxford Archaeology Digital
  Janus House
  Osney Mead
  OX2 0ES
  Oxford, U.K.
 
  Tel: +44 (0)1865 263 800 (switchboard)
  Tel: +44 (0)1865 980 758 (direct)
  Fax :+44 (0)1865 793 496
  benjamin.du...@oadigital.net
  http://oadigital.net
 
 
 
 
 
  --
  Files attached to this email may be in ISO 26300 format (OASIS Open
  Document Format). If you have difficulty opening them, please visit
  http://iso26300.info for more information.
 
  ___
  grass-user mailing list
  grass-user@lists.osgeo.org
  http://lists.osgeo.org/mailman/listinfo/grass-user
 
  ___
  grass-user mailing list
  grass-user@lists.osgeo.org
  http://lists.osgeo.org/mailman/listinfo/grass-user



-- 
Dylan Beaudette
Soil Resource Laboratory
http://casoilresource.lawr.ucdavis.edu/
University

Re: [GRASS-user] Searching Docs about 3D geological modelisation

[Dylan Beaudette]

Hi Thomas,

I am pretty sure that the program (maybe not the source code) are in
the public domain.

I'll contact the author, and post back.

Cheers,
Dylan

On Mon, Jan 11, 2010 at 6:56 AM, Thomas Adams thomas.ad...@noaa.gov wrote:
 Dylan,

 Can you tell me how to obtain TPROGS? Is it only available commercially?

 Thanks,
 Tom


 Dylan Beaudette wrote:

 Two more ideas:

 1. conditional simulation, based on a 3D variogram model
 2. transition probability-based interpolation of categories

 Check out gstat for the conditional simulation, and TPROGS for the
 transition probability. If anything is interested, I have done some
 programming to connect GRASS and TPROGS.

 Cheers!

 Dylan

 On Fri, Jan 8, 2010 at 1:24 PM, Benjamin Ducke
 benjamin.du...@oxfordarch.co.uk wrote:


 Woohoo, this forum is always a treasure trove
 of good advice. I had not idea SGemS existed!
 The Voronoi idea is also good, I am just not sure
 that the 3D Voronoi diagram is quite what one
 would instinctively think it is.

 http://en.wikipedia.org/wiki/Voronoi_diagram

 says: In general a cross section of a 3D Voronoi
 tessellation is not a 2D Voronoi tessellation itself.

 Need to look into that.

 I don't have much practical experience
 with Bayes models, so can't really comment on
 that.

 Cheers,

 Ben


 Christian Kaiser wrote:


 It seems to me that this is a 3D interpolation problem with categorical
 variables.

 Maybe the Bayesian Maximum Entropy approach could help. There are some
 interesting publications around also for geology and soil sciences, and 
 they
 can deal with categorical data as well. Look for example here:
 http://www.enge.ucl.ac.be/staff/curr/Bogaert/biblioBME/BMEbibsubject.html#Soil%20Science

 Or maybe you can have a look at SGeMS (http://sgems.sourceforge.net), a
 tool for 3D geostatistics.

 None of them is available through GRASS, but the algorithms are freely
 available (I think open-source, but not verified).

 I am not a geologist, so please forgive if it is not adequate...

 Christian Kaiser





 On 8 janv. 2010, at 11:04, Benjamin Ducke wrote:



 Rich Shepard wrote:


 material. There is no interpolation algorithm in GRASS currently
 which
 can
 handle that sort of data well.


  So what is needed is a political algorithm. :-)


 That's actually right: given the presence of n different
 layer types in the vicinity of an empty voxel, the algorithm
 would need to decide by some sort of majority vote
 which type to assign to that voxel.



  Kidding aside, I suspect that a fuzzy interpolation algorithm would
 solve
 the problem.


 How? You could make the interpolated value depend on a
 fuzzy set member function, I suppose, but the situation
 here is actually so well defined that I think a probabilistic
 approach would be preferable. Since each voxel can only
 store one value, a second output map could store the
 classification probability. That may be very useful
 for visualization (you could show voxels with little
 probability hazier).

 Ben



 Rich
 ___
 grass-user mailing list
 grass-user@lists.osgeo.org
 http://lists.osgeo.org/mailman/listinfo/grass-user




 --
 Benjamin Ducke
 Geospatial Consultant

 Oxford Archaeology Digital
 Janus House
 Osney Mead
 OX2 0ES
 Oxford, U.K.

 Tel: +44 (0)1865 263 800 (switchboard)
 Tel: +44 (0)1865 980 758 (direct)
 Fax :+44 (0)1865 793 496
 benjamin.du...@oadigital.net
 http://oadigital.net





 --
 Files attached to this email may be in ISO 26300 format (OASIS Open
 Document Format). If you have difficulty opening them, please visit
 http://iso26300.info for more information.

 ___
 grass-user mailing list
 grass-user@lists.osgeo.org
 http://lists.osgeo.org/mailman/listinfo/grass-user


 ___
 grass-user mailing list
 grass-user@lists.osgeo.org
 http://lists.osgeo.org/mailman/listinfo/grass-user




 --
 Benjamin Ducke
 Geospatial Consultant

 Oxford Archaeology Digital
 Janus House
 Osney Mead
 OX2 0ES
 Oxford, U.K.

 Tel: +44 (0)1865 263 800 (switchboard)
 Tel: +44 (0)1865 980 758 (direct)
 Fax :+44 (0)1865 793 496
 benjamin.du...@oadigital.net
 http://oadigital.net





 --
 Files attached to this email may be in ISO 26300 format (OASIS Open
 Document Format). If you have difficulty opening them, please visit
 http://iso26300.info for more information.

 ___
 grass-user mailing list
 grass-user@lists.osgeo.org
 http://lists.osgeo.org/mailman/listinfo/grass-user



 ___
 grass-user mailing list
 grass-user@lists.osgeo.org
 http://lists.osgeo.org/mailman/listinfo/grass-user



 --
 Thomas E Adams
 National Weather Service
 Ohio River Forecast Center
 1901 South State Route 134
 Wilmington, OH 45177

 EMAIL:  thomas.ad...@noaa.gov

 VOICE:  937-383-0528
 FAX:    937-383-0033


___
grass

Re: [GRASS-user] Searching Docs about 3D geological modelisation

[Dylan Beaudette]

On Sat, Jan 9, 2010 at 2:51 AM, Benjamin Ducke
benjamin.du...@oxfordarch.co.uk wrote:
 Cheers for these. They are certainly all highly interesting.
 Do you have an actual link for the T-PROGS software itself?
 All I can seem to come up with are interfaces from other
 software and publications mentioning it.

 I would certainly be interested in taking a look at your
 GRASS interface. Is T-PROGS open source?

 My gut feeling is that the T-PROGS approach would give better
 results than 3D kriging, as it seems better able to to
 follow 3D shape trends:

 http://chl.erdc.usace.army.mil/chl.aspx?p=sa=ARTICLES;37g=50

 ... but that certainly would need testing.

 Having said that, I also like this approach for a more
 heuristic model:

 http://chl.erdc.usace.army.mil/chl.aspx?p=sa=ARTICLES;41g=50

 It's very simple and could easily be implemented directly
 in GRASS GIS. In fact, I coded something very similar to this
 for archaeological stratigraphy reconstruction a while back.

 Cheers,

 Ben

Hi Ben,

Yes. It would be very interesting to have these functions within GRASS
libraries, as opposed to the kludgy interfacing that I did via shell
scripting + awk. Here are some of the details, from *several* years
ago (GRASS 5.x):

http://169.237.35.250/~dylan/grass_and_tp/

... note that this is rather old work, and somethings may have changed
since then.

Here is the reference for the software:

Carle, Steven F. T-PROGS Transition Probability Geostatistical
Software Version 2.1 manual. University of California, Davis. 1999.

I can get in touch with Graham Fogg here at UC Davis, whom I believe
is in charge of maintaining the current implementation of T-PROGS--
basically fortran source + a tcl/tk interface. Having this
functionality in GRASS would greatly add to the capabilities of the
voxel framework.

Also, by 'conditional simulation' I wasn't referring to kriging per
se, rather the conditional simulation of an indicator (categorical)
variable, based on random fields + variogram model. the gstat library
can perform both unconditional simulation (randomness only tied to a
variogram model), and conditional simulation (randomness tied to real
point data + variogram model).

I'll report back here with my findings.

Cheers,
Dylan





 - Original Message -
 From: Dylan Beaudette dylan.beaude...@gmail.com
 To: Benjamin Ducke benjamin.du...@oxfordarch.co.uk
 Cc: GRASS user list grass-user@lists.osgeo.org
 Sent: Saturday, January 9, 2010 4:30:40 AM GMT +01:00 Amsterdam / Berlin / 
 Bern / Rome / Stockholm / Vienna
 Subject: Re: [GRASS-user] Searching Docs about 3D geological modelisation

 Two more ideas:

 1. conditional simulation, based on a 3D variogram model
 2. transition probability-based interpolation of categories

 Check out gstat for the conditional simulation, and TPROGS for the
 transition probability. If anything is interested, I have done some
 programming to connect GRASS and TPROGS.

 Cheers!

 Dylan

 On Fri, Jan 8, 2010 at 1:24 PM, Benjamin Ducke
 benjamin.du...@oxfordarch.co.uk wrote:
 Woohoo, this forum is always a treasure trove
 of good advice. I had not idea SGemS existed!
 The Voronoi idea is also good, I am just not sure
 that the 3D Voronoi diagram is quite what one
 would instinctively think it is.

 http://en.wikipedia.org/wiki/Voronoi_diagram

 says: In general a cross section of a 3D Voronoi
 tessellation is not a 2D Voronoi tessellation itself.

 Need to look into that.

 I don't have much practical experience
 with Bayes models, so can't really comment on
 that.

 Cheers,

 Ben


 Christian Kaiser wrote:
 It seems to me that this is a 3D interpolation problem with categorical 
 variables.

 Maybe the Bayesian Maximum Entropy approach could help. There are some 
 interesting publications around also for geology and soil sciences, and 
 they can deal with categorical data as well. Look for example here: 
 http://www.enge.ucl.ac.be/staff/curr/Bogaert/biblioBME/BMEbibsubject.html#Soil%20Science

 Or maybe you can have a look at SGeMS (http://sgems.sourceforge.net), a 
 tool for 3D geostatistics.

 None of them is available through GRASS, but the algorithms are freely 
 available (I think open-source, but not verified).

 I am not a geologist, so please forgive if it is not adequate...

 Christian Kaiser





 On 8 janv. 2010, at 11:04, Benjamin Ducke wrote:

 Rich Shepard wrote:
 material. There is no interpolation algorithm in GRASS currently which
 can
 handle that sort of data well.
  So what is needed is a political algorithm. :-)
 That's actually right: given the presence of n different
 layer types in the vicinity of an empty voxel, the algorithm
 would need to decide by some sort of majority vote
 which type to assign to that voxel.

  Kidding aside, I suspect that a fuzzy interpolation algorithm would solve
 the problem.
 How? You could make the interpolated value depend on a
 fuzzy set member function, I suppose, but the situation
 here is actually so well defined that I think

Re: [GRASS-user] Searching Docs about 3D geological modelisation

[Dylan Beaudette]

Two more ideas:

1. conditional simulation, based on a 3D variogram model
2. transition probability-based interpolation of categories

Check out gstat for the conditional simulation, and TPROGS for the
transition probability. If anything is interested, I have done some
programming to connect GRASS and TPROGS.

Cheers!

Dylan

On Fri, Jan 8, 2010 at 1:24 PM, Benjamin Ducke
benjamin.du...@oxfordarch.co.uk wrote:
 Woohoo, this forum is always a treasure trove
 of good advice. I had not idea SGemS existed!
 The Voronoi idea is also good, I am just not sure
 that the 3D Voronoi diagram is quite what one
 would instinctively think it is.

 http://en.wikipedia.org/wiki/Voronoi_diagram

 says: In general a cross section of a 3D Voronoi
 tessellation is not a 2D Voronoi tessellation itself.

 Need to look into that.

 I don't have much practical experience
 with Bayes models, so can't really comment on
 that.

 Cheers,

 Ben


 Christian Kaiser wrote:
 It seems to me that this is a 3D interpolation problem with categorical 
 variables.

 Maybe the Bayesian Maximum Entropy approach could help. There are some 
 interesting publications around also for geology and soil sciences, and they 
 can deal with categorical data as well. Look for example here: 
 http://www.enge.ucl.ac.be/staff/curr/Bogaert/biblioBME/BMEbibsubject.html#Soil%20Science

 Or maybe you can have a look at SGeMS (http://sgems.sourceforge.net), a tool 
 for 3D geostatistics.

 None of them is available through GRASS, but the algorithms are freely 
 available (I think open-source, but not verified).

 I am not a geologist, so please forgive if it is not adequate...

 Christian Kaiser





 On 8 janv. 2010, at 11:04, Benjamin Ducke wrote:

 Rich Shepard wrote:
 material. There is no interpolation algorithm in GRASS currently which
 can
 handle that sort of data well.
  So what is needed is a political algorithm. :-)
 That's actually right: given the presence of n different
 layer types in the vicinity of an empty voxel, the algorithm
 would need to decide by some sort of majority vote
 which type to assign to that voxel.

  Kidding aside, I suspect that a fuzzy interpolation algorithm would solve
 the problem.
 How? You could make the interpolated value depend on a
 fuzzy set member function, I suppose, but the situation
 here is actually so well defined that I think a probabilistic
 approach would be preferable. Since each voxel can only
 store one value, a second output map could store the
 classification probability. That may be very useful
 for visualization (you could show voxels with little
 probability hazier).

 Ben

 Rich
 ___
 grass-user mailing list
 grass-user@lists.osgeo.org
 http://lists.osgeo.org/mailman/listinfo/grass-user



 --
 Benjamin Ducke
 Geospatial Consultant

 Oxford Archaeology Digital
 Janus House
 Osney Mead
 OX2 0ES
 Oxford, U.K.

 Tel: +44 (0)1865 263 800 (switchboard)
 Tel: +44 (0)1865 980 758 (direct)
 Fax :+44 (0)1865 793 496
 benjamin.du...@oadigital.net
 http://oadigital.net





 --
 Files attached to this email may be in ISO 26300 format (OASIS Open 
 Document Format). If you have difficulty opening them, please visit 
 http://iso26300.info for more information.

 ___
 grass-user mailing list
 grass-user@lists.osgeo.org
 http://lists.osgeo.org/mailman/listinfo/grass-user


 ___
 grass-user mailing list
 grass-user@lists.osgeo.org
 http://lists.osgeo.org/mailman/listinfo/grass-user




 --
 Benjamin Ducke
 Geospatial Consultant

 Oxford Archaeology Digital
 Janus House
 Osney Mead
 OX2 0ES
 Oxford, U.K.

 Tel: +44 (0)1865 263 800 (switchboard)
 Tel: +44 (0)1865 980 758 (direct)
 Fax :+44 (0)1865 793 496
 benjamin.du...@oadigital.net
 http://oadigital.net





 --
 Files attached to this email may be in ISO 26300 format (OASIS Open Document 
 Format). If you have difficulty opening them, please visit 
 http://iso26300.info for more information.

 ___
 grass-user mailing list
 grass-user@lists.osgeo.org
 http://lists.osgeo.org/mailman/listinfo/grass-user

___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] Mask creation from sporadic data v.hull

[Dylan Beaudette]

Hi,

I have solved this problem in the past with a combination of g.region,
v.to.rast, and r.grow.

For example, if you import your points in as a vector-- convert to
raster, increase the cell size, and grow out a couple of cells. If you
are bringing in your lidar data with r.in.xyz, then it is already a
raster, and you can adjust the resolution and grow accordingly. Make
sense?

Cheers,
Dylan

On Fri, Dec 18, 2009 at 4:04 AM, John Tate john.t...@ntlworld.com wrote:
 Hi all,

 For a while now I have been trying to make an accurate mask for LiDAR points.
 As I am working with 1m resolution data I want the mask to be accurate (no
 more than 1m around the edge of the points/flight area).

 The data is wavy around the edges, due to a windy flight when the data was
 collected, so v.hull ignores the waves and joins the crests, adding vast NULL
 areas to the mask. Also, there are up to 10m holes within the individual
 swaths.

 r.in.xyz at 5m resolution has given me the best results so far as final
 accuracy masks, but not for the processing as I want holes within the data
 filled with v.surf.rst.

 I have looked at v.delaunay and v.voronoi too, but both need topology built
 and don't have similar outputs to v.hull regarding vast NULL areas. This is ok
 for 1km tiles, but I want to process each swath separately (study on swath
 accuracy) and these can be c.5million points each.

 Does anyone know if there is way to do this accurately without manually
 digitizing? Have I missed something in the book/manual?

 Cheers

 John

 ___
 grass-user mailing list
 grass-user@lists.osgeo.org
 http://lists.osgeo.org/mailman/listinfo/grass-user

___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] Old GRASS GIS compilation

[Dylan Beaudette]

On Monday 23 November 2009, Glynn Clements wrote:
 Firman Hadi wrote:
  I am compiling GRASS 4.3 on Debian Woody.
  My aim is to use r.le.dist function. I have tried to compile it but there
  is one error that I can't solve. The error message is like below:
 
  /home/firman/grass4.3src/src/libes/vask
   make -f OBJ/make.rules
 
  rm -f OBJ/V_call.o
  gcc   -I/home/firman/grass4.3src/src/include   -DUSE_TERMIO -c V_call.c
  V_call.c: In function 'V_call':
  V_call.c:355: warning: comparison between pointer and integer
  V_call.c: At top level:
  V_call.c:430: error: static declaration of 'fmt' follows non-static
  declaration
  V_call.c:171: error: previous implicit declaration of 'fmt' was here
  V_call.c:449: error: static declaration of 'centered' follows non-static
  declaration
  V_call.c:234: error: previous implicit declaration of 'centered' was here
  make: *** [OBJ/V_call.o] Error 1
  GISGEN failure at STEP: src/libes/vask
 
  By this email, I would like to ask help from anyone who have experience
  installing GRASS 4.3.

 You can fix these specific errors by moving the definitions of fmt()
 and centered() from the bottom of the file to the top.

 But you may encounter many more such errors trying to compile GRASS
 4.3 with a modern C compiler. You could try using an older compiler,
 but that may not work with the headers from modern libraries.


Hi,
Any reason not to use the current version of GRASS?

Dylan

-- 
Dylan Beaudette
Soil Resource Laboratory
http://casoilresource.lawr.ucdavis.edu/
University of California at Davis
530.754.7341
___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

[GRASS-user] spatial connectivity function

[Dylan Beaudette]

Greetings,

We are trying to calculate hydrologic connectivity for a 33ha catchment based 
on a an interpolated raster from a series of 100 spatially distributed soil 
moisture measurements. Ideally, we would like to use the methods outlined (in 
pseudo-code) here:

ftp://ftp.agu.org/apend/wr/2000WR900241/Psuedo_code.txt

It looks like step 1 (thresholding the data) can be done with r.mapcalc. Step 
2 (label the connected regions) can be accomplished with r.clump. Now we are 
stuck on step 3a (calculate the omnidirectional connectivity function) which 
is some kind of estimate of linear 'connectivity'. Are there any modules that 
can approximate this type of analysis, or is this going to require a new 
module?

Here is a sample dataset that approximates what we are working with

# generate a fake, spatially-correlated field
r.surf.fractal out=fract dimension=2.8

# convert to indicators
r.mapcalc fract.ind = if(fract  0, 1, 2)

# form spatially-connected 'clumps'
r.clump in=fract.ind out=fract.clump

Cheers,
Dylan


___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] Strategy For DBMS Use

[Dylan Beaudette]

On Wednesday 18 November 2009, Rich Shepard wrote:
 On Wed, 18 Nov 2009, Pablo Carreira wrote:
  I have very nice results with sqlite+GRASS.

 Pablo,

I've been using SQLite for several years; it's the back end to my
 approximate reasoning model for environmental impact assessments.

  First, use an sqlite administrator software to make things easier.

I'm a CLI guy and prefer working by typing.

  One database for each LOCATION, one table for each vector.

That's what I thought. Thanks for confirming.

I like this setup as well. However I have noticed one major shortcoming of 
this approach: when a long-running vector operation is using the SQLite 
database, no other processes can access vector attributes. I noticed this 
when performing a large v.patch job. No other vector modules that opened the 
attribute table would work, or would break with errors. Maybe there is 
something that can be done to solve this, however, it seems like the 
single-user nature of SQLite is to blame. One SQLite database per vector 
seems like a heavy-handed solution, but it would solve some of the 
problems--- while creating others...

Cheers,
Dylan


  One way of working is that You can put rivers, highways, power lines,
  etc. in the same layer and put a column in the table that describe what
  kind of object it is. You can play with CAT number too, like rivers is
  cat number x and highways is cat number y.

I'll need to think about this. Right now I have three locations, based
 on regional extent. Most layers are for the entire state, the DEM is for
 the northwest portion of the state and the soils are for a single county.

  Try no to think as the obsolete shapefile style gis.

I've never used ARC/View. I did use ARC/Info on the Prime in the late
 1980s, and a bit of ARC/Info-PC in the same period. Most of the mapping I
 did in the 1990s was with MapInfo running under VMware on a linux host. I
 last upgraded that in 1996. :-) Haven't done any GIS work for about 6
 years.

  I am a agronomist engineer an I have made soil maps and soil use maps
  with grass, just ask if you have more questions.

And I'm a stream ecologist/fluvial geomorphologist so I'm sure I'll have
 questions about antecedent soild moisture for the model I'm working toward
 building.

 Thanks,

 Rich
 ___
 grass-user mailing list
 grass-user@lists.osgeo.org
 http://lists.osgeo.org/mailman/listinfo/grass-user



-- 
Dylan Beaudette
Soil Resource Laboratory
http://casoilresource.lawr.ucdavis.edu/
University of California at Davis
530.754.7341
___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] how to find peaks

[Dylan Beaudette]

On Wednesday 18 November 2009, Michael Barton wrote:
 Beyond r.param.scale, is there a good method anyone knows of to find
 peaks or hilltops? I'm more interested in the tops of hills/high
 points than the single cell that is the highest.

 Thanks
 Michael

Hi,

The r.param.scale uses some common cuttoffs for curvatures to define features. 
A geomorphic classification may accomplish what you are looking for, i.e. 
generate some training areas that define 'summits', through in several 
terrain-shape indices, and then run a supervised classification. Unsupervised 
classification can sometimes work well for this kind of task, however it is 
data-driven and results can vary from location to location.

Cheers,
Dylan



 
 C. Michael Barton
 Director, Center for Social Dynamics  Complexity
 Professor of Anthropology, School of Human Evolution  Social Change
 Arizona State University

 Phone: 480-965-6262
 Fax: 480-965-7671
 www: www.public.asu.edu/~cmbarton, http://csdc.asu.edu







 ___
 grass-user mailing list
 grass-user@lists.osgeo.org
 http://lists.osgeo.org/mailman/listinfo/grass-user



-- 
Dylan Beaudette
Soil Resource Laboratory
http://casoilresource.lawr.ucdavis.edu/
University of California at Davis
530.754.7341
___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

[GRASS-user] r.drain indir option

[Dylan Beaudette]

Hi,

Is it possible to substitute a 'direction' raster generated from a
module other than r.cost, when using the 'indir' option for r.drain?
This is for GRASS65.

Specifically, I would like to compute a least-cost path based on a
friction surface AND direction map. Is this possible?

Thanks,
Dylan
___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] v.to.rast with text column

[Dylan Beaudette]

On Wednesday 14 October 2009, Hamish wrote:
 Hamish:
  maybe I am missing something, but why not just do:
  v.to.rast use=cat labelcolumn=name ?

 sorry I was still thinking about ticket #175,
 that should read:

  v.to.rast use=attr column=cat labelcolumn=name


 H

OK-- Yes. That is what I did. However, I cancelled the job before it 
completed, as it was taking 10 times longer than without the label column. I 
found that dumping my lookup table from postgres, reformatting with awk, and 
then labeling cats with r.category was much faster.

Cheers,
Dylan

-- 
Dylan Beaudette
Soil Resource Laboratory
http://casoilresource.lawr.ucdavis.edu/
University of California at Davis
530.754.7341
___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] v.to.rast with text column

[Dylan Beaudette]

Great! Thanks for the tips. I was able to accomplish this (in PostGresql) with 
the following steps:

1. generate unique list of text categories as a temp table
2. add a serial column to the temp table (auto-incrementing integer)
3. add new column to original table
4. UPDATE original table with the new serial number, assigned by text category
5. v.to.rast

Cheers,
Dylan


On Wednesday 14 October 2009, Markus Neteler wrote:
 IMHO ticket 175 has been fixed. I can now run (thanks to Martin):

 v.db.select lsat7_training
 cat|name|id
 1|water|1
 3|forest|2
 4|forest|2
 5|asphalt|3
 6|asphalt|3
 7|water|1
 ...
 v.to.rast in=lsat7_training out=lsat7_training use=attr col=id
 labelcol=name

 As Achim says: col must be numerical and should be the numerical
 representation of the text attributes (see example above).

 Markus

 On Wed, Oct 14, 2009 at 1:39 PM, Hamish hamis...@yahoo.com wrote:
  see also  https://trac.osgeo.org/grass/ticket/175
 
  Achim wrote:
  v.to.rast has the option:
 
  --%--
  labelcolumn=name
      Name of column used as raster category
  labels
  --%--
 
  And the raster values have to be numeric:
 
  --%--
  column=name
      Name of column for attr parameter (data type
  must be numeric)
  --%--
 
  (http://grass.itc.it/grass64/manuals/html64_user/v.to.rast.html)
 
  Try to make unique numbers from your string-column and
  convert them with
  column=... and labelcolumn...
 
  Achim
 
  Dylan Beaudette schrieb:
   Hi,
  
   v.to.rast is letting me know that it does not accept
 
  text columns in an
 
   attribute table. Has anyone figured out a simple
 
  approach to making this
 
   work? I suppose that I could import the data into R,
 
  convert to factors, save
 
   the levels back to the table and use those... but
 
  there may be something out
 
   there that is simpler and can be done within GRASS.
 
  Also, would this be
 
   something worth adding to the v.to.rast code?
  
   Cheers,
   Dylan
 
  ___
  grass-user mailing list
  grass-user@lists.osgeo.org
  http://lists.osgeo.org/mailman/listinfo/grass-user



-- 
Dylan Beaudette
Soil Resource Laboratory
http://casoilresource.lawr.ucdavis.edu/
University of California at Davis
530.754.7341
___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] v.to.rast with text column

[Dylan Beaudette]

On Wednesday 14 October 2009, Dylan Beaudette wrote:
 Great! Thanks for the tips. I was able to accomplish this (in PostGresql)
 with the following steps:

 1. generate unique list of text categories as a temp table
 2. add a serial column to the temp table (auto-incrementing integer)
 3. add new column to original table
 4. UPDATE original table with the new serial number, assigned by text
 category 5. v.to.rast

 Cheers,
 Dylan

One small note-- I noticed that the operation takes MUCH longer to complete 
when the labelcolumn argument is specified. So long in fact, that I canceled 
the process, and converted only the integer representation to raster. I then 
added the category labels with r.category. Something in v.to.rast was causing 
the postgresql instance to peg the CPU at 100% when specifying a labelcolumn. 
This was for a rather large vector conversion, about 410520 features.

Cheers,
Dylan


 On Wednesday 14 October 2009, Markus Neteler wrote:
  IMHO ticket 175 has been fixed. I can now run (thanks to Martin):
 
  v.db.select lsat7_training
  cat|name|id
  1|water|1
  3|forest|2
  4|forest|2
  5|asphalt|3
  6|asphalt|3
  7|water|1
  ...
  v.to.rast in=lsat7_training out=lsat7_training use=attr col=id
  labelcol=name
 
  As Achim says: col must be numerical and should be the numerical
  representation of the text attributes (see example above).
 
  Markus
 
  On Wed, Oct 14, 2009 at 1:39 PM, Hamish hamis...@yahoo.com wrote:
   see also  https://trac.osgeo.org/grass/ticket/175
  
   Achim wrote:
   v.to.rast has the option:
  
   --%--
   labelcolumn=name
       Name of column used as raster category
   labels
   --%--
  
   And the raster values have to be numeric:
  
   --%--
   column=name
       Name of column for attr parameter (data type
   must be numeric)
   --%--
  
   (http://grass.itc.it/grass64/manuals/html64_user/v.to.rast.html)
  
   Try to make unique numbers from your string-column and
   convert them with
   column=... and labelcolumn...
  
   Achim
  
   Dylan Beaudette schrieb:
Hi,
   
v.to.rast is letting me know that it does not accept
  
   text columns in an
  
attribute table. Has anyone figured out a simple
  
   approach to making this
  
work? I suppose that I could import the data into R,
  
   convert to factors, save
  
the levels back to the table and use those... but
  
   there may be something out
  
there that is simpler and can be done within GRASS.
  
   Also, would this be
  
something worth adding to the v.to.rast code?
   
Cheers,
Dylan
  
   ___
   grass-user mailing list
   grass-user@lists.osgeo.org
   http://lists.osgeo.org/mailman/listinfo/grass-user



-- 
Dylan Beaudette
Soil Resource Laboratory
http://casoilresource.lawr.ucdavis.edu/
University of California at Davis
530.754.7341
___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

[GRASS-user] v.to.rast with text column

[Dylan Beaudette]

Hi,

v.to.rast is letting me know that it does not accept text columns in an 
attribute table. Has anyone figured out a simple approach to making this 
work? I suppose that I could import the data into R, convert to factors, save 
the levels back to the table and use those... but there may be something out 
there that is simpler and can be done within GRASS. Also, would this be 
something worth adding to the v.to.rast code?

Cheers,
Dylan

-- 
Dylan Beaudette
Soil Resource Laboratory
http://casoilresource.lawr.ucdavis.edu/
University of California at Davis
530.754.7341
___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] clear sky index interpolation

[Dylan Beaudette]

Hi,

Are you trying to generate a 2D raster map of interpolated values, or
a 3D 'volume' of interpolated values? If your input consists of
{lat,lon,kc}, then a 2D raster map is likely what you are after. In
that case, check out the v.surf.rst module.

Cheers,
Dylan

On Fri, Sep 4, 2009 at 6:49 AM, buraqzak...@gmail.com wrote:



 buraq wrote:

 Hi,
 It's a newbie question.
 I am trying to intepolate clear sky index values. First I put the values
 in a csv file in this format lat,lon,kc.
 Then I used db.in.ogr to get table. Next step, vector raster map is
 created by v.in.db. When I want to create raster file, I use that command
 v.vol.rst input=Kc vect cellinp=elevationRaster wcolumn=indexValueColumn
 then I get this Vector is not 3D
 Should I add elevation to csv? or what is the right way the interpolate
 the data?

 Thanks
 Regards



 hi again I tried interpolation with elevation data.  Now I got x,y and z But
 now the problem is, v.vol.rst gives the Error in COGRR! Interp_call
 failed message. Have you ever meet this kind of a error message? I searched
 the list but couldn't find any information about that kind of problem.

 Thanks
 Regards
 --
 View this message in context: 
 http://n2.nabble.com/clear-sky-index-interpolation-tp3572894p3580073.html
 Sent from the Grass - Users mailing list archive at Nabble.com.
 ___
 grass-user mailing list
 grass-user@lists.osgeo.org
 http://lists.osgeo.org/mailman/listinfo/grass-user

___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] r.sun problem

[Dylan Beaudette]

All this recent talk about bugs in r.sun / r.sun2 has made me a bit
concerned about recent research built on r.sun. Should I be concerned
about incorrect results from r.sun when supplied with an aspect +
slope map, as run on GRASS 6.4, about 2 years ago?

Thanks,

Dylan

On Sat, Aug 22, 2009 at 6:56 AM, Hamishhamis...@yahoo.com wrote:
 Hi,

 I am using r.sun -s to calculate solarmaps for each month.
 I wrote a shell script which calculates every 10th day
 (including the shaddowing effect). But something is going
 wrong. I looks like the shaddows are displaced somehow (see
 attachement). Using r.sun2 -s didnt make any difference.
 Curiously once it worked! But although I'm using the same
 projection, location, mapset and DSM - it doesnt work
 anymore.

 I hope anybody could help me out or have any ideas how I
 could solve this problemI really need these maps.
 .
 I am using Quantum Gis Version 1.0.2 and GRASS 6.4.0 svn


 The region  of my investigation area is:

 $ g.region -p
 projection: 99 (Transverse Mercator)
 zone:       0
 datum:
 towgs84=577.326,90.129,463.919,5.137,1.474,5.297,2.4232
 ellipsoid:  bessel
 north:      281981.20714553
 south:      281612.20714553
 west:       -29172.52857137
 east:       -28383.52857137
 nsres:      1
 ewres:      1
 rows:       369
 cols:       789
 cells:      291141


 could you provide the exact command line you were using?
 are you using r.horizon seeds, or slope and aspect seed maps?
 what time step?

 By chance I noticed in xpdf if I dragged with the left mouse
 button instead of the middle one (to pan) I got an inverse
 image which made the effect a bit clearer. With that I notice
 that the center-southmost  building in the ok map actually
 isn't.

 how do you run the two different versions? did you compile the
 old one? is this the latest SVN? (like younger than a week?)
 is QGIS only used for visualization or do you run grass from
 the toolbox?

 lately we have been running some tests with it*, but in general
 I think the new version (ie really upgrade) is in fact better
 than the old, if just because you can avoid the slope/aspect
 bug now.

 [*] see http://grass.osgeo.org/wiki/r.sun
 and https://trac.osgeo.org/grass/ticket/498



 building shadows is something I'd always wanted to try with
 r.sun, interesting to see some results! May I ask if it is
 LIDAR elevations or simply by e.g. number of storeys?

 also, may I suggest r.colors map color=grey -e, where the -e
 flag may help mute the contrasts a bit.


 Hamish





 ___
 grass-user mailing list
 grass-user@lists.osgeo.org
 http://lists.osgeo.org/mailman/listinfo/grass-user

___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] Geodatabase and GRASS GIS - how to get the data out of a geodatabase

[Dylan Beaudette]

I have used 'mdbtools' in the past for this type of operation.
However, this will only extract attribute data-- not geometry data. I
think that you will need a windows build of GDAL/ORG to accomplish
that. Check the GDAL page.

Here are some examples of using the mdbtools package:
http://casoilresource.lawr.ucdavis.edu/drupal/node/462

Cheers,
Dylan

On Sun, Aug 9, 2009 at 8:19 AM, William
Kyngesburyewokl...@kyngchaos.com wrote:
 Markus,

 I was trying to get mdbtools working on OSX this weekend (yet another
 attempt), and thought your R method might be an alternative.  But that also
 requires mdb-tools - that's where mdb-tables tool is found.

 Agustin: If I remember right, you're on OSX?  If so, you are out of luck, I
 have yet to find or compile a working MDB library for OSX.

 On Aug 9, 2009, at 9:45 AM, Agustin Diez Castillo wrote:

 I've tried the R solution w/o success, see below

 On Aug 9, 2009, at 8:35 AM, Markus Neteler wrote:

 On Sun, Aug 9, 2009 at 3:23 AM, stephen sefickssef...@gmail.com wrote:

 I have a GIS database that is in .mdb format (microsoft access).  I
 have found mdb tools through my debian package manager, but now I
 don't know how to get the data out of the the database and into GRASS.
  There are x and y coordinates and other things in the database.  Is
 there a tutorial on how to get the data into GRASS from this database?

 I see two ways:

 1) via unixODBC
 For Linux, you need unixODBC to be installed (your preferred
 distribution
 most likely contains this package). As done, get also the MDB Tools,
 compile it with unixodbc support enabled if not already done.
 Then follow instructions at http://www.gdal.org/ogr/drv_pgeo.html

 2) via R

 R
 install.packages('Hmisc', dependencies=TRUE)
 library(Hmisc)
 # Read all tables in the Microsoft Access database blabla.mdb
 #   perhaps better change to Locale in which the .mdb file was
 #   generated to avoid that accented table column names are
 #   getting lost.file first
 d - mdb.get('/path/to/blabla.mdb')
 contents(d)
 for(z in 1:length(d)) print(attributes(d)$names[z])

 d - mdb.get('/path/to/blabla.mdb')
 returns to me with
 sh: mdb-tables: command not found

 -
 William Kyngesburye kyngchaos*at*kyngchaos*dot*com
 http://www.kyngchaos.com/

 Mon Dieu! but they are all alike.  Cheating, murdering, lying, fighting,
 and all for things that the beasts of the jungle would not deign to possess
 - money to purchase the effeminate pleasures of weaklings.  And yet withal
 bound down by silly customs that make them slaves to their unhappy lot while
 firm in the belief that they be the lords of creation enjoying the only real
 pleasures of existence

 - the wisdom of Tarzan


 ___
 grass-user mailing list
 grass-user@lists.osgeo.org
 http://lists.osgeo.org/mailman/listinfo/grass-user

___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] Interactive flood simulation

[Dylan Beaudette]

On Wednesday 05 August 2009, Luca Casagrande wrote:
 Pavel Iacovlev wrote:
  Good day,
 
  I wrote a simple web flood simulation application. Posting here the
  source code, maybe it will be useful for someone who wants to use
  GRASS from PHP.

Nice work!

-- 
Dylan Beaudette
Soil Resource Laboratory
http://casoilresource.lawr.ucdavis.edu/
University of California at Davis
530.754.7341
___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] r.stream: new features

[Dylan Beaudette]

On Saturday 01 August 2009, Jarek Jasiewicz wrote:
 Dear GRASS Users!

 I announce new features of r.stream: Horton stream order. More details
 and actual source code:

 http://heretic.livenet.pl/heretic/

 The release of full version is planned at the fall of August.

 Test and enjoy.

 Jarek

Nice work Jarek!

All options work well on some of my data. I noticed that the module segfaults 
if the 'accum' input is not given and the 'horton' or 'shreeve' output 
options are set.

Cheers,
Dylan

-- 
Dylan Beaudette
Soil Resource Laboratory
http://casoilresource.lawr.ucdavis.edu/
University of California at Davis
530.754.7341
___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] Re: Building topologically correct stream network

[Dylan Beaudette]

On Tuesday 21 July 2009, Jarek Jasiewicz wrote:
 BTW I'm about to finish regular grass modue for calculating Strahler
 Stream order and Shreeve Stream Magnitude. I hope I publish it for
 testing in next few days. Any help appreciated.

I would be happy to help test!

Cheers,
Dylan

-- 
Dylan Beaudette
Soil Resource Laboratory
http://casoilresource.lawr.ucdavis.edu/
University of California at Davis
530.754.7341
___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] shapefile, TIGER, or what for a dlg that is stored in .e00 format

[Dylan Beaudette]

On Wed, Jul 8, 2009 at 10:17 PM, Hamishhamis...@yahoo.com wrote:

 stephen sefick wrote:
  I am trying to make a DEM from contour lines downloaded from here
  http://csat.er.usgs.gov/statewide/layers/contours.html

 fwiw, v.in.ogr's SDTS driver might help to import DLGs more directly.
 ??


 Nikos wrote:
 Perhaps you do not even need to rasterise. Have a look at v.surf.rst
 [2]. Of course I am no expert with DEM's, v.surf.rst might not be what
 you need.

 v.surf.rst does not do all that well with contour lines. due to the
 adaptive grid size / quadtree design it focuses detail on where the data 
 points are. In this case that's the vertices along the contour lines.

 r.surf.contour does a nice job with them though, just read in the help
 page about overcoming the lack of floating-point support, if that is
 needed.


 also, as mentioned there is probably a higher resolution DEM already out
 there for free, e.g. SRTM 1 (~30m) resolution or USGS quads.

Right. On the seamless.usgs.gov server you should be able to get the
most recent 10m resolution DEM for free in GeoTiff or Arc Grid format.

Cheers,
Dylan
___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] .e00 to DEM

[Dylan Beaudette]

On Wednesday 08 July 2009, Nikos Alexandris wrote:
 There is something strange with the values though (they expand from min
 = -32687  max = 32645). Don't have the time to dig further, maybe there
 are details in the meta-data about it (!?).

This looks like an overflow problem. Could it be that this file contains 
unsigned 16 / 32 bit integers, but is being read in as signed 16 bit 
integers?

This used to happen when reading in Arc ASCII grids that contained very large 
numbers ( 32768) somewhere other than in the first couple of lines. I am 
pretty sure this bug has been fixed-- but I am not sure how GDAL is 
interpreting this specific file. It would be useful to use gdal_translate to 
copy the file, forcing the bit-size of the output to something else.

Here was the original GRASS ticket: http://trac.osgeo.org/grass/ticket/166
and the subsequent GDAL ticket: http://trac.osgeo.org/gdal/ticket/2369

There are some tips in the GDAL ticket on forcing a file to be read as a 32bit 
integers (etc.). Quoth Frank W:

Note that with gdal_translate you can convert pixel types on the fly, so if 
you know the data range is suitable for Int16, you could 
do gdal_translate -ot Int16 in.grd out.tif for instance.

Cheers,
Dylan



-- 
Dylan Beaudette
Soil Resource Laboratory
http://casoilresource.lawr.ucdavis.edu/
University of California at Davis
530.754.7341
___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] .e00 to DEM

[Dylan Beaudette]

On Wednesday 08 July 2009, Nikos Alexandris wrote:
 On Wed, 2009-07-08 at 09:51 -0700, Dylan Beaudette wrote:
  On Wednesday 08 July 2009, Nikos Alexandris wrote:
   There is something strange with the values though (they expand from min
   = -32687  max = 32645). Don't have the time to dig further, maybe there
   are details in the meta-data about it (!?).
 
  This looks like an overflow problem. Could it be that this file contains
  unsigned 16 / 32 bit integers, but is being read in as signed 16 bit
  integers?

 Probably you are right Dylan. In the meta-data it is written:

  Level 2 DEM: Level 2 DEMs may contain void areas due to interruptions
 to contours in the source graphic or DLG. Void area elevation grid posts
 are assigned the value of -32,767. 

 * This means that original data are for sure Signed (probably
 Int16-bit... ?).

 * gdal reads the data correctly (look previous post of mine) as:
   Band 1 Block=20x4 Type=Int16, ColorInterp=Undefined
   Min=0.000 Max=1371.000
   NoData Value=-32768

 * grass reports the range of the imported data as:
 r.info dem250 -tr
 min=-32687
 max=32645
 datatype=CELL

 * The data show up correctly in GRASS:

 g.region rast=dem250  r.colors dem250 color=terrain  d.mon x0 
 d.rast dem250


 If we accept min=-32687 that was assigned to be nodata, what is this
 max=32645?

 Nikos

Hi Nikos, 

Looks like a 16bit signed integer file. I have found that in the past using 
gdal_translate and manually setting the data type and nodata value results in 
the generation of a new file that GRASS can read in without further work. 
Also, have you tried manually setting NULL cells with 

r.null setnull=32768

Sometimes GRASS isn't notified of the nodata upon importing...

Dylan

-- 
Dylan Beaudette
Soil Resource Laboratory
http://casoilresource.lawr.ucdavis.edu/
University of California at Davis
530.754.7341
___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] shapefile, TIGER, or what for a dlg that is stored in .e00 format

[Dylan Beaudette]

Hi,

Do you need to use this data for the generation of a DEM? Would it be
possible to use another source? seamless.usgs.gov is a great place to
get gridded elevation data for the USA. Interpolating from contours
should be a last resort.

Cheers,
Dylan

On Wed, Jul 8, 2009 at 6:42 PM, stephen sefickssef...@gmail.com wrote:
 I tried this with points, lines and areas on the .e00 file

 v.in.e00 'file=/Users/sefick/Desktop/contours Folder/contours.e00'
 type=area vect=georgia_contours --overwrite

 importing areas..

 unable to open data source cont
 an error occured. Stop.

 what now?

 I didn't have any luck with the grid data - tried and gave up.

 thanks

 Stephen Sefick

 On Wed, Jul 8, 2009 at 7:38 PM, Nikos
 Alexandrisnikos.alexand...@felis.uni-freiburg.de wrote:

 stephen sefick wrote:

 I am trying to make a DEM from contour lines downloaded from here
 http://csat.er.usgs.gov/statewide/layers/contours.html
 I converted this to a shape file

 You don't need to convert it to  Shapefile. You can import vector .e00
 files directly in GRASS using v.in.e00. Note that you need to have to
 programs installed: avce00 and e00compr.


 reprojected it,

 why? from what to what CRS? Did you not create a location based on the
 coordinate reference system in which the data are referenced? Did you
 have any success with the GRID data, if of course you tried?


  and then v.to.rast use=value and got out a raster with a range of 1
 to 1.

 That is so because the v.to.rast module expects from the user to define
 the value incase you use the use=value parameter. If the user does
 not define the value then value=1 is taken as default. Please read
 the respective manual(s) [1].

 I suppose that v.to.rast use=val value=SomeValue is not what you want.
 Giving a fixed value to all of the vector features that will be
 rasterized wont be useful if you want to play further with the data
 (e.g. create a DEM as you mention above).


 How do I do this?

 --%---code--%---
 # I downloaed the data you mention and did the following:
 v.in.e00 contours.e00 vect=contours type=line

 # check attribute table
 v.info -c contours

 v.info -c contours
 Displaying column types/names for database connection of layer 1:
 INTEGER|cat
 INTEGER|UserId
 INTEGER|FNODE_
 INTEGER|TNODE_
 INTEGER|LPOLY_
 INTEGER|RPOLY_
 DOUBLE PRECISION|LENGTH
 INTEGER|CONTOURS_
 INTEGER|CONTOURS_I
 INTEGER|ELEV

 # match region ## I am unsure about the resolution (=look at the
 original data resolution from which the contours derived)
 g.region vect=contours res=SomeResolutionValue -pa

 # the last column is probably of your interest, so
 v.to.rast use=val value=attr col=ELEV
 --%---code--%---

 Perhaps you do not even need to rasterise. Have a look at v.surf.rst
 [2]. Of course I am no expert with DEM's, v.surf.rst might not be what
 you need.

 Kind regards, Nikos
 ---

 [1] http://grass.osgeo.org/grass64/manuals/html64_user/v.to.rast.html

 [2] http://grass.osgeo.org/grass64/manuals/html64_user/v.surf.rst.html





 --
 Stephen Sefick

 Let's not spend our time and resources thinking about things that are
 so little or so large that all they really do for us is puff us up and
 make us feel like gods.  We are mammals, and have not exhausted the
 annoying little problems of being mammals.

                                                                -K. Mullis
 ___
 grass-user mailing list
 grass-user@lists.osgeo.org
 http://lists.osgeo.org/mailman/listinfo/grass-user

___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] AVIRIS data in GRASS

[Dylan Beaudette]

On Monday 06 July 2009, Michael Perdue wrote:
 I was able to generate a ENVI style header that will allow you to read
 the files with GDAL. Modify as necessary and you should be able to
 work with the *.img files.

Fantastic. Thanks for the tips Mike. I will give this a try. For the record, 
Frank W. over on the GDAL mailing list suggested the code samples below. A 
script could be used to generate a single file for all 224 bands.

# make one of these for the entire set of data
# add more VRTRasterBand elements for the bands you want:
#
VRTDataset rasterXSize=614 rasterYSize=512
   VRTRasterBand dataType=Int16 band=1 subClass=VRTRawRasterBand
 SourceFilename
relativetoVRT=1f970619t01p02_r02_sc01.c.img/SourceFilename
 ImageOffset0/ImageOffset
 PixelOffset448/PixelOffset
 LineOffset275072/LineOffset
 ByteOrderMSB/ByteOrder
   /VRTRasterBand
/VRTDataset

# multi-band example:
VRTDataset rasterXSize=614 rasterYSize=512
   VRTRasterBand dataType=Int16 band=1 subClass=VRTRawRasterBand
 SourceFilename
relativetoVRT=1f970619t01p02_r02_sc01.c.img/SourceFilename
 ImageOffset0/ImageOffset
 PixelOffset448/PixelOffset
 LineOffset275072/LineOffset
 ByteOrderMSB/ByteOrder
   /VRTRasterBand
   VRTRasterBand dataType=Int16 band=2 subClass=VRTRawRasterBand
 SourceFilename
relativetoVRT=1f970619t01p02_r02_sc01.c.img/SourceFilename
 ImageOffset2/ImageOffset
 PixelOffset448/PixelOffset
 LineOffset275072/LineOffset
 ByteOrderMSB/ByteOrder
   /VRTRasterBand
/VRTDataset

This works well in QGIS-- and I imagine in GRASS as well. The tricky part from 
here is applying the rectification and navigation parameters...

Cheers,
Dylan


 On 5-Jul-09, at 6:56 PM, Michael Perdue wrote:
  ... you might be able to manipulate GDAL to read it. If you can
  create a header file for a generic binary file that will provide
  GDAL with the number of rows, columns, byte order, #channels, # of
  bytes/sample and the interleave method then you can tell GDAL how to
  read it.
 
  On 5-Jul-09, at 3:49 PM, Dylan Beaudette wrote:
   I
  have tried working with the radiance data, which when uncompressed
  comes with several .img files-- however my copy of GDAL does not know
  what to make of it. It looks like it may be possible [2] to import
  the
  data using r.in.bin... I was not able to get this working.

 Cheers,

 Mike



-- 
Dylan Beaudette
Soil Resource Laboratory
http://casoilresource.lawr.ucdavis.edu/
University of California at Davis
530.754.7341
___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

[GRASS-user] AVIRIS data in GRASS

[Dylan Beaudette]

Hi,

I have been searching mailing lists for a while, and have not been
able to discover any approaches for working with AVIRIS data in GRASS.
Does anyone on the list know how to either convert or extract data as
delivered in the free data section [1] of the AVIRIS home page? I
have tried working with the radiance data, which when uncompressed
comes with several .img files-- however my copy of GDAL does not know
what to make of it. It looks like it may be possible [2] to import the
data using r.in.bin... I was not able to get this working.



1. http://aviris.jpl.nasa.gov/html/aviris.freedata.html
2. http://n2.nabble.com/AVIRIS---grass-td1864545.html

Thanks!

Dylan
___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] Re: link to datasets from other locations?

[Dylan Beaudette]

On Thu, Jul 2, 2009 at 12:07 AM, Hamishhamis...@yahoo.com wrote:
 2c POV:
 *every* single GIS I have seen which does on-the-fly reprojection has had
 major problems which can present itself in a way which is not obvious
 to the user (typically due to mixed datums). Then the user happily goes
 along doing their work, publishing their reports, only sometimes
 realizing that something has misaligned internally. Or ugly non-solutions
 like rubber sheet dragging to visually correct the misalignment without
 considering the real cause become taught as the de facto method in the
 workflow. (easier than teaching about ellipsoids and datum math)

 I admit it's not the same as theoretically it could be done correctly,
 but I sort of class it with the common request for scalebars or 1:xx,xxx
 map scale for lat/lon cartography. It sounds like a good idea, and lots
 of other software give the users what they demand there, but that doesn't
 mean it's valid or a good idea. For map projections it is not invalid,
 just really really hard to get right (probably because right is often
 a very subjective answer in geodetics).


 Hamish

Yes. I am probably preaching to the choir here, but I have seen this
time and time again in both GIS classes, along with faculty, making
these type of basic (but not obvious) mistakes. Seems like a good
idea, but usually results in confusing and mysterious errors when
processing the data.

Dylan
___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] hardware requirements for running r.sun

[Dylan Beaudette]

On Tuesday 30 June 2009, Hamish wrote:
 buraq wrote:
  I have a notebook with T7200 Intel CPU and r.sun completes
  one day's calculation between 3-4 mins(with linke turbidity raster
  and shadowing, step = 1). I want to run r.sun with step=0.5 in a
  large scale and detailed area.
  So, I need to reduce the calculation time for calculating 365 day data.
  Could you give me advices about hardware for reducing
  calculation time?

 see tips here:
   http://grass.osgeo.org/wiki/r.sun

 (run r.horizon first)

 24 hours to complete isn't so bad :)
 suggestion: to take advantage of dual processor split up operations and
 (if running linux) run two concurrent mapsets for day 0-180 and 181-365.
 then when done use g.mapsets to add second mapset to first's search path.

 wish: someone in the know adds OpenMP* support to r.sun
 [*] (or pthreads, but OpenMP seems easier)
  http://grass.osgeo.org/wiki/OpenMP


 Hamish


Not trying to honk one's own horn, but an application of r.sun and time 
requirements was recently published here:

http://soil.scijournals.org/cgi/content/abstract/73/4/1345

Let me know if you want a PDF

Dylan




-- 
Dylan Beaudette
Soil Resource Laboratory
http://casoilresource.lawr.ucdavis.edu/
University of California at Davis
530.754.7341
___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] predict.glm and predict.gam output

[Dylan Beaudette]

On Thursday 18 June 2009, Anke Konrad wrote:
 Hi all,

 I am currently trying to compare different plant occurrence prediction
 maps generated in R and exported into GRASS. One of these maps was
 generated from a glm fitted to some data, and subsequently applying this
 glm model to a wider region using predict.glm. The outcome here was a
 probability of occurrence. The second map I generated using a gam
 (mgcv), however, this map seems to have assigned something like a
 negative log-likelihood of occurrence to each raster cell in the region.
 Since I would like to compare the two, I would like to figure out a way
 of having the same kind of output from the predict functions (either
 probability OR negative log-likelihood). Does anyone know of a way of
 changing the output options? And if not, does anyone have any
 suggestions of how I could deal with this issue?


Hi,

This is actually a question for the r-help mailing list. If you can provide 
some code samples, along with the type of model you are trying to fit (sounds 
like logistic regression model), I bet you will get some good answers. 

Usual suspects when you get bad output from a predict.*() function:
1. you fed it data that was not in the same format as the original data used 
to fit the model
2. you fed it data that was not what you thought it was: i.e. a map read in 
from GRASS as the wrong datatype, ect.
3. you fed it data that was way outside the range of data used to fit the 
model

As always be sure to check ?predict.gam and ?predict.glm .

Cheers,
Dylan


-- 
Dylan Beaudette
Soil Resource Laboratory
http://casoilresource.lawr.ucdavis.edu/
University of California at Davis
530.754.7341
___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] Calculating line length over terrain

[Dylan Beaudette]

On Thursday 11 June 2009, Hamish wrote:
  Falk Grossmann wrote:
   My objective is to calculate the length of a line (GPS tracklog) in a
   very mountainous terrain. In this case I will need to take into
   account the 3D aspect.

 be aware this is very much a fractal problem with no true answer.

Here are some examples that may be of interest:

http://casoilresource.lawr.ucdavis.edu/drupal/node/375

Cheers,
Dylan

-- 
Dylan Beaudette
Soil Resource Laboratory
http://casoilresource.lawr.ucdavis.edu/
University of California at Davis
530.754.7341
___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] Density of points in a vector layer

[Dylan Beaudette]

See v.kernel

Cheers,
Dylan

On Wed, Jun 10, 2009 at 7:42 AM, Paolo Tomepaolo_t...@hotmail.com wrote:

 Hello,
 I'm using grass in combination with qgis; I would know how can I
 calculate the density of points in a vector layer. In the past I used
 gvsig and Sextante, in particular I used  density kernel tool. Which
 equivalent application can I use in Grass?
 Thank you for any useful advice!
 Paolo Tomè



 ___
 grass-user mailing list
 grass-user@lists.osgeo.org
 http://lists.osgeo.org/mailman/listinfo/grass-user

___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] Density of points in a vector layer

[Dylan Beaudette]

On Wed, Jun 10, 2009 at 8:27 AM, Paolo Tomepaolo_t...@hotmail.com wrote:
 Thanks for your reply, I tried to use v.kernel but the result is not
 very satisfactory (a raster uniformly colored of gray), I don't have
 much familiarity with the console, then I use the graphical interface of
 the application (using QGIS) . Then I can set only the maps in
 Stdeviation units (parameter which I did not understand the full
 meaning). Can you give me some guidance on how to obtain a better
 result? Thank you very much.

Hi Paolo,

The command line is the most flexible approach to using GRASS. When
using modules that generate raster output you need to consider the
region resolution and extent. Have a look at the raster intro
(http://grass.itc.it/grass64/manuals/html64_user/rasterintro.html).
You will need to work out a resolution that makes sense, then set it
with g.region. Finally, you will have to tinker around with the stddev
option to v.kernel for results that look right to you. Think of this
parameter as the diameter of the 'filter' in which point densities are
calculated. The size of this parameter will be a function of your
dataset and the region parameters.

This module could use an updated manual page, as its usage is somewhat
mysterious.

Cheers,
Dylan



 On mer, 2009-06-10 at 08:18 -0700, Dylan Beaudette wrote:
 See v.kernel

 Cheers,
 Dylan

 On Wed, Jun 10, 2009 at 7:42 AM, Paolo Tomepaolo_t...@hotmail.com wrote:
 
  Hello,
  I'm using grass in combination with qgis; I would know how can I
  calculate the density of points in a vector layer. In the past I used
  gvsig and Sextante, in particular I used  density kernel tool. Which
  equivalent application can I use in Grass?
  Thank you for any useful advice!
  Paolo Tomè
 
 
 
  ___
  grass-user mailing list
  grass-user@lists.osgeo.org
  http://lists.osgeo.org/mailman/listinfo/grass-user
 




___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] exponentiation with v.db.update

[Dylan Beaudette]

Hi,

Which database back-end are you using? I seem to recall that the DBF
back-end only supports a very limited subset of update operations.

cheers,
Dylan

On Tue, Jun 9, 2009 at 8:56 AM, A. Marcia
BARBOSAanamarciabarb...@gmail.com wrote:
 Hi,
 I'm trying to populate a column with 10 raised to the power of the
 values in another column:

 v.db.update map=ItaReg column=x qcolumn=10^logx

 but I get a dbmi protocol error. I've tried using either single or
 double quotes with no success. If I use a combination of both, i.e.
 qcolumn= ' 10^logx '  it doesn't report any error, but doesn't do
 the operation either (the column remains unchanged).

 It works fine for other operations (e.g. 10*logx), though. Is there
 another way in which I should specify the exponentiation?

 Thanks,
 Marcia
 ___
 grass-user mailing list
 grass-user@lists.osgeo.org
 http://lists.osgeo.org/mailman/listinfo/grass-user

___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] exponentiation with v.db.update

[Dylan Beaudette]

On Tuesday 09 June 2009, A. Marcia BARBOSA wrote:
  why not using sqlite backend? dbf is very limited for any serious use.

 It may seem quite simple for some, but to me all these things are a
 complete mystery... I'm actually very surprised I became able to
 handle GRASS! I'm planning on switching all my databases to PostgreSQL
 within this year (after taking a course, of course), but right now I
 have to work with what I've got... dbf it is. I've just managed to
 calculate the column outside and re-connect it to the vector map.


Well, when you get a chance, here are some tips on converting DB back-ends:
http://casoilresource.lawr.ucdavis.edu/drupal/node/733

Note that there is a lot of red text on that page. It means be careful!.

Cheers,
Dylan


-- 
Dylan Beaudette
Soil Resource Laboratory
http://casoilresource.lawr.ucdavis.edu/
University of California at Davis
530.754.7341
___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] Averaging multiple vector lines

[Dylan Beaudette]

On Thu, Jun 4, 2009 at 9:41 AM, Dylan Beaudette
dylan.beaude...@gmail.com wrote:
 On Wed, Jun 3, 2009 at 6:23 PM, Dwight Needels need...@translucida.com 
 wrote:
 I have a GRASS vector that originated as multiple GPS tracks from walking a
 particular trail segment on several different days. Is there a good way to
 average these lines to get a single line? I want to minimize GPS accuracy
 errors by averaging across multiple days and also minimize precision errors
 (random jumping around on a single day) while still maintaining the shape of
 the trail with all of its twists and turns.

 I have been able to generate a composite vector by using a combination of
 v.to.rast, r.grow, r.thin, r.to.vect, v.clean, and v.generalize
 method=douglas. This method works pretty well when the lines remain close
 together, but it is very dependent on picking a value for the r.grow radius
 that fills in all of the gaps between the multiple tracks. If one track is
 quite different than the others in even a single region of the vector, this
 requires a relatively large radius value. Moreover, the final vector is
 located about midway between the two extremes rather than being weighted
 toward where the majority of tracks fall.

 It seems like there would be a way to calculate some sort of sliding average
 of the coordinates that fall within a certain size window, perhaps after
 using v.to.points with a small dmax (5 ft?) to generate a fairly dense set
 of points. Ideally, the calculation window could be wider perpendicular to
 the direction of the line than it is along the direction of the line. From
 day to day tracks are often within 10 to 20 ft of each other, but it is not
 uncommon for two tracks to be 30 ft away from each other at some points.

 Any ideas?

 -Thanks, -Dwight


 I have often wanted to do something like this with GPS tracks, however
 I have never thought to try your vector-raster-vector approach -- very
 creative!

 I think that a vector-based approach could be implemented along the
 lines you mention:

 1. v.to.points on each GPS track
 2. v.patch to  collect all points into single vector
 3. new module to generate an average 'centerline' along the cloud of points.

 This last approach could be done fairly nicely in cartesian space with
 a smoothing algorithm such as lowess or supersmooth.

 Here is an example in R, graphic attached.

 # densified collection of points along a single GPS track
 x - rnorm(n=100, mean=1, sd=0.1)

 # check
 plot(x, type='b')

 # generate 10 densified GPS tracks, based on our original track
 m - x + matrix(rnorm(n=1000, mean=0.25, sd=0.05), ncol=10)

 # check
 matplot(m, type='l', lty=1, col=1, ylab='y-coordinate', xlab='x-coordinate')

 # convert from wide to long format, as dataframe
 d - data.frame(x=rep(1:100, 10), y=as.vector(m))

 # compute lowess smooth, and plot as red line
 lines(lowess(d$x, d$y, f=0.01), col='red', lwd=2)


 So, it may be possible to augment the v.generalize command to work
 with a collection of nodes (i.e. accept multiple vector inputs). Or,
 an implementation of the lowess algorithm would be another approach.

 Cheers,
 Dylan


Ack.. I just realized that this won't work if the trail crosses over
itself, or where two x-coordinate values occur at a single
y-coordinate or visa versa. un-raveling the trails along some linear
path would be required to apply the lowess smoother. v.generalize or
Michael's suggestion may be the best approach.

Cheers,
Dylan
___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] Can this be done in GRASS

[Dylan Beaudette]

On Friday 22 May 2009, william told wrote:
 Hi, the attached 3D image I believe was generated by software. I was
 wondering is someone could tell me if there is a way to create something
 similar with GRASS, out of shapefiles (or do I need something more, like an
 aerial photo?) Many thanks
 Bill


Hi,

Looks like some kind of thematic map, draped over an elevation surface, with a 
vector representation of a river system on the right-side. 

This is not at all difficult to do in GRASS, you just need some data first. Do 
you have 

1. thematic data in raster / vector format?
2. data sufficient to generate an elevation surface, or a DEM?
3. hydrologic data in raster / vector format?

Once you have these things, all in the same coordinate system- planar if 
possible, bring them into a grass mapset (see the manual). You can generate 
perspectives like the attached image with a tool called 'nviz'. Here is an 
example, using a DEM, aerial photo, and 3D tree objects (see the GRASS 
add-ons page):

http://169.237.35.250/~dylan/temp/pinn_trees_rnd.jpg

Cheers,
Dylan



-- 
Dylan Beaudette
Soil Resource Laboratory
http://casoilresource.lawr.ucdavis.edu/
University of California at Davis
530.754.7341
___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] traveling salesman problem in air

[Dylan Beaudette]

On Wednesday 15 April 2009, Vincent Bain wrote:
 Don't know if it makes sense, but I would try v.delaunay to build a net
 between closest centroids (given that each centroid is roughtly centered
 on its area).

 Vincent

That seems like a good judgement call, considering that the resulting network 
of connecting every centroid could become very large. 

This problem reminded me of a similar task, where we wanted to visit data 
loggers in the field and were not constrained to any existing 'network'. I 
found that a cost-surface (in this case accumulated slope gradient) gave us a 
nice constraint on how to use a simple network derived from v.delaunay. Some 
notes on that operation here:

http://casoilresource.lawr.ucdavis.edu/drupal/node/698

Cheers,
Dylan

-- 
Dylan Beaudette
Soil Resource Laboratory
http://casoilresource.lawr.ucdavis.edu/
University of California at Davis
530.754.7341
___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] Off Topic: Importing GRASS tiffs to GMT

[Dylan Beaudette]

On Tue, Apr 7, 2009 at 2:58 AM, John Stevenson
john.steven...@manchester.ac.uk wrote:
 Hamish wrote:

 Eric wrote:


 I'll post here before checking on the GMT list;
 hopefully I can get some answers from a GRASS/GMT guru...


 Moritz wrote:


 I can't help you on your precise questions, but have you seen
 Dylan's article:
 http://casoilresource.lawr.ucdavis.edu/drupal/node/561
 (see link to pdf on that page) ?


Which of course should be updated


 and of course the GRASS Wiki:
  http://grass.osgeo.org/wiki/GMT

 (which of course forever needs updating)

Sigh... on my to-do list, really!


 By a tiff, do you mean a georeferenced map? I've plotted aerial photos that
 I rectified in GRASS using the following to export the RGB bands:

 r.mapcalc image.red=r#image; image.green=g#image; image.blue=b#image
 r.out.bin -h input=image.red output=image.red.grd
 r.out.bin -h input=image.green output=image.green.grd
 r.out.bin -h input=image.blue output=image.blue.grd

 Followed by:

 grdimage image.red.grd image.green.grd image.blue.grd -J -R -B ...etc.

 I've just put the same workflow onto the wiki.  I've also used r.his to make
 coloured shaded relief maps, and plotted them in GMT using the same method.
  I don't think that it is the optimal method, but at least it preserved my
 colour rules.

 Cheers

 John


Yep. This is nealry the same general approach that I use, however- I
tend to pipe the output from r.out.bin to  xyz2grd.

Here is a script with a pile of commands that may be useful to some:
http://169.237.35.250/~dylan/temp/where_2.0/template.sh

... note that it could use some updating as well.

Cheers,
Dylan
___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] Error in sqlite3_step()

[Dylan Beaudette]

Yeah. I have recently noticed this type of behavior after switching to
sqlite. I was unable to patch two vectors (and att tables) after
defining new columns of type varchar(20). I re-made the columns as
'text' and v.patch worked as expected. Maybe we need some special,
internal conversions (varchar --- text) when working with sqlite db.

Cheers,

Dylan


On Mon, Apr 6, 2009 at 2:58 AM, Moritz Lennert
mlenn...@club.worldonline.be wrote:
 On 06/04/09 11:41, Moritz Lennert wrote:

 On 03/04/09 17:06, Craig Leat wrote:

 Moritz Lennert wrote:

 Don't have time to look into this now, but it must be a problem in the
 special sqlite implementation in that script:

 if [ $driver = sqlite ] ; then
       #echo Using special trick for SQLite
       # http://www.sqlite.org/faq.html#q13
       v.info --q -c map=$GIS_OPT_MAP layer=$GIS_OPT_LAYER | cut -d'|'
 -f1,2
 | grep -v |${col}$  $TMP.coldesc
       # need to revert order:
       cat $TMP.coldesc | cut -d'|' -f1  $TMP.coltypes
       cat $TMP.coldesc | cut -d'|' -f2  $TMP.colnames
       COLDEF=`paste -d' ' $TMP.colnames $TMP.coltypes | tr '\n' ','
 |
 sed 's+,$++g'`
       COLNAMES=`cat $TMP.colnames | tr '\n' ',' | sed 's+,$++g'`
 echo BEGIN TRANSACTION;
 CREATE TEMPORARY TABLE ${table}_backup(${COLDEF});
 INSERT INTO ${table}_backup SELECT ${COLNAMES} FROM ${table};
 DROP TABLE ${table};
 CREATE TABLE ${table}(${COLDEF});
 INSERT INTO ${table} SELECT ${COLNAMES} FROM ${table}_backup;
 DROP TABLE ${table}_backup;
 COMMIT;  $TMP
 db.execute input=$TMP

 My execute statement looks like this:
 BEGIN TRANSACTION;
 CREATE TEMPORARY TABLE SBC2_backup(cat INTEGER,LM CHARACTER,WARD
 INTEGER,CLASS_NAME CHARACTER);
 INSERT INTO SBC2_backup SELECT cat,LM,WARD,CLASS_NAME FROM SBC2;
 DROP TABLE SBC2;
 CREATE TABLE SBC2(cat INTEGER,LM CHARACTER,WARD INTEGER,CLASS_NAME
 CHARACTER);
 INSERT INTO SBC2 SELECT cat,LM,WARD,CLASS_NAME FROM SBC2_backup;
 DROP TABLE SBC2_backup;
 COMMIT;

 Why does v.db.dropcol define a type CHARACTER when sqlite3 supports
 (NULL, INTEGER, REAL, TEXT, BLOB)?

 v.db.dropcol just uses the output of v.info -c, and AFAIR, SBC2 has its
 columns defined as CHARACTER, or ?

 See also this thread:

 http://lists.osgeo.org/pipermail/grass-user/2009-January/048281.html

 Moritz
 ___
 grass-user mailing list
 grass-user@lists.osgeo.org
 http://lists.osgeo.org/mailman/listinfo/grass-user

___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

[GRASS-user] minor annoyance with sqlite db driver

[Dylan Beaudette]

I recently converted several of my mapsets over to the sqlite datebase driver. 

I noticed that when a long-running process (such as v.surf.rst) happens to 
write to a vector attribute table at the same time some other process tries 
to access and attribute table (d.vect ... where=...) the first process dies 
saying that the database is locked. 

Has this behavior been documented elsewhere? Anyone else notice this type of 
situation?

Cheers,
Dylan

-- 
Dylan Beaudette
Soil Resource Laboratory
http://casoilresource.lawr.ucdavis.edu/
University of California at Davis
530.754.7341
___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] convert r.report output to spreadsheet format

[Dylan Beaudette]

On Wed, Feb 25, 2009 at 1:39 AM, Giovanni Pasini jynx...@gmail.com wrote:
 maning sambale wrote:

 Thanks!
 An example of the r.report output:

 |-|
 |              Category Information               |          |     cell|
 %  |
 | #|description                                   |  hectares|    count|
 cover|

 |-|
 | 1|Siquijor                                      |    31,313|   385511|
  0.05|
 |
  |--|--|-|--|
 |  |112|Broadleaf - open canopy. . . . . . . . . .|      8265|   101755|
 26.39|
 |  |140|Mangrove . . . . . . . . . . . . . . . . .|       173|     2125|
  0.55|
 |  |210|Agriculture-annual . . . . . . . . . . . .|      7370|    90737|
 23.54|
 |  |310|Scrub. . . . . . . . . . . . . . . . . . .|      1063|    13081|
  3.39|
 |  |320|Grassland. . . . . . . . . . . . . . . . .|      7185|    88460|
 22.95|
 |  |400|Built-up . . . . . . . . . . . . . . . . .|       103|     1270|
  0.33|
 |  |510|Inland water . . . . . . . . . . . . . . .|        44|      539|
  0.14|
 |  |  *|no data. . . . . . . . . . . . . . . . . .|      7111|    87544|
 22.71|

 I want converted to:
 Province,land_cover_code,description,hectares,cell_count,percent_cover

 I will try the cryptic awk lines.


 On Wed, Feb 25, 2009 at 12:12 AM, Giovanni Pasini jynx...@gmail.com
 wrote:

 maning sambale ha scritto:

 Hi,

 Looking for tips in transforming r.report output into a
 spreadsheet like format.

 Hi,
 I use awk (change NF and the other fields $*):

 r.report -n map=gri_1000_in,dem2_idw_100_slope_rec units=h,p,c | awk
 'BEGIN
 {FS=|;OFS=|} !/---/  NF==7 {intest=$2} NF==8 {sub(/[ .]*$/,,$4);
 print intest, $4, $6}'  report_slope.txt

 This from a non awk expert!

 cheers,
 giovanni
 ___
 grass-user mailing list
 grass-user@lists.osgeo.org
 http://lists.osgeo.org/mailman/listinfo/grass-user





 try this:

 cat report.txt |awk 'BEGIN {FS=|;OFS=|} !/---/  NF==7 {intest=$3}
 NF==8 {sub(/[ .]*$/,,$4); print intest, $4,$5,$6,$7}'

 or better:

 cat report.txt | awk 'BEGIN {FS=|;OFS=|} !/---/  NF==7 {intest=$3}
 NF==8 {sub(/[ .]*$/,,$4); sub(/[ ]*$/,,intest); printf
 %s|%s|%d|%d|%.2f\n,intest,$4,$5,$6,$7}'


 giovanni


Just opened a new feature request ticket:
http://trac.osgeo.org/grass/ticket/507

I may be able to get around to working on this mid april... :( Unless
someone else wants to tackle it!

Cheers,
Dylan
___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] improve v.rast.stats speed?

[Dylan Beaudette]

On Thursday 19 February 2009, G. Allegri wrote:
 Hi Dylan.
 I didn't let it finish because 15 minutes were too many for my task.
 Ok, less then 5 hours and more of v.rast.stats, but too much respect
 to ArcGIS and the rasterization solution in GRASS.
 I've built the 1.2.03 version, downloaded from [1].
 Anyway I suspect the same about GRASS driver inefficiencies in GDAL/OGR

 [1]
 http://projects.atlas.ca.gov/frs/download.php/667/starspan-1.2.03.tar.gz

OK. This is the old stable branch (I think). If you can get 2.0 to compile I 
would suggest trying that. Starspan really needs to make it into OSGeo so 
that more eyes can get in on the development + bug tracking. At one point it 
was considerably faster than zonal stats in ArcGIS. I am planning on spending 
more time on Starspan from May.

Cheers,

Dylan


 2009/2/19 Dylan Beaudette dylan.beaude...@gmail.com:
  On Thu, Feb 19, 2009 at 5:20 AM, G. Allegri gioha...@gmail.com wrote:
  Thanks for the ideas.
  I've just tried Starspan but it's performance is still too slow. I've
  let it run for 15 minutes...
 
  Hi,
 
  Did you ever let it finish? Can you post the version number? I have
  noticed that starspan tends to be slower when using GRASS vector and
  raster features-- probably a combination of inefficiencies in GDAL/OGR
  with the GRASS formats.
 
 
  Dylan
 
  r.statistics is probably the best solution. I've investigated the
  ArcGIS method and it actually seems to use a similar method
  (ratserization of the features and various automations to join the
  results). In fact they call the module zonal statistics that is
  generally a set of raster basded methods.
 
  the only limitation of the actual r.statistics is that it works only
  with CELL and not float. Ok, I can multiply my values and convert to
  CELL, but we could try to let r.statistics deal with floats too...
 
  I will try to batch the process and let you know the results.
 
  2009/2/19 Markus Metz markus.metz.gisw...@googlemail.com:
  Markus Metz wrote:
  G. Allegri wrote:
  Hello list.
  Yesterday I needed to use v.rast.stats on a 1793 areas covering a
  4415x6632 raster (with resolution 50m/pixel). I've used it without
  extended statistics but the processing time was, with an euphemism,
  very very long. After 5 hours it wasn't finished yet. As I needed it
  for today morning I've decided to reproduce it with ArcGIS: 40
  seconds. I've tried to investigate what was going wrong, the
  bottleneck, but at the end I suppose that it's a problem of the
  script itself (the looping chain of r.mapcalc and r.univar, the
  creation and deletion of the MASK in each loop).
  Is there any way to improve the performance of v.rast.stats? Should
  we rewrite it in C and avoid the use of MASKs?
 
  I have two ideas.
  1) Use r.reclass instead of r.mapcalc to create new masks. That should
  speed up at least the MASK creation and deletion
  2) Avoid the loop and MASK creation altogether. Run r.univar
  map=tmpname,raster. Process the output of r.univar, separate stats for
  the different vector areas and convert to sql statements. Proceed as
  before. r.univar would be called only once. I'm not sure if this is
  possible. I also don't know if the speed gain by avoiding the loop is
  annihilated by r.univar having to process two rasters as input.
 
  Idea 2 is nonsense, I hoped for some behaviour like in r.statistics.
 
  ___
  grass-user mailing list
  grass-user@lists.osgeo.org
  http://lists.osgeo.org/mailman/listinfo/grass-user



-- 
Dylan Beaudette
Soil Resource Laboratory
http://casoilresource.lawr.ucdavis.edu/
University of California at Davis
530.754.7341
___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] improve v.rast.stats speed?

[Dylan Beaudette]

On Thu, Feb 19, 2009 at 5:20 AM, G. Allegri gioha...@gmail.com wrote:
 Thanks for the ideas.
 I've just tried Starspan but it's performance is still too slow. I've
 let it run for 15 minutes...

Hi,

Did you ever let it finish? Can you post the version number? I have
noticed that starspan tends to be slower when using GRASS vector and
raster features-- probably a combination of inefficiencies in GDAL/OGR
with the GRASS formats.


Dylan



 r.statistics is probably the best solution. I've investigated the
 ArcGIS method and it actually seems to use a similar method
 (ratserization of the features and various automations to join the
 results). In fact they call the module zonal statistics that is
 generally a set of raster basded methods.

 the only limitation of the actual r.statistics is that it works only
 with CELL and not float. Ok, I can multiply my values and convert to
 CELL, but we could try to let r.statistics deal with floats too...

 I will try to batch the process and let you know the results.

 2009/2/19 Markus Metz markus.metz.gisw...@googlemail.com:


 Markus Metz wrote:

 G. Allegri wrote:

 Hello list.
 Yesterday I needed to use v.rast.stats on a 1793 areas covering a
 4415x6632 raster (with resolution 50m/pixel). I've used it without
 extended statistics but the processing time was, with an euphemism,
 very very long. After 5 hours it wasn't finished yet. As I needed it
 for today morning I've decided to reproduce it with ArcGIS: 40
 seconds. I've tried to investigate what was going wrong, the
 bottleneck, but at the end I suppose that it's a problem of the script
 itself (the looping chain of r.mapcalc and r.univar, the creation and
 deletion of the MASK in each loop).
 Is there any way to improve the performance of v.rast.stats? Should we
 rewrite it in C and avoid the use of MASKs?


 I have two ideas.
 1) Use r.reclass instead of r.mapcalc to create new masks. That should
 speed up at least the MASK creation and deletion
 2) Avoid the loop and MASK creation altogether. Run r.univar
 map=tmpname,raster. Process the output of r.univar, separate stats for the
 different vector areas and convert to sql statements. Proceed as before.
 r.univar would be called only once. I'm not sure if this is possible. I also
 don't know if the speed gain by avoiding the loop is annihilated by r.univar
 having to process two rasters as input.

 Idea 2 is nonsense, I hoped for some behaviour like in r.statistics.


 ___
 grass-user mailing list
 grass-user@lists.osgeo.org
 http://lists.osgeo.org/mailman/listinfo/grass-user

___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] Thiessen Polygons

[Dylan Beaudette]

On Wednesday 11 February 2009, MORREALE Jean Roc wrote:
 Hamish a écrit :
  Kurt Springs wrote:
  Does anyone know how to do Thiessen Polygons in GRASS?
 
  I need to use them around various types of megalithic tomb
  sites that are in point vector files.
 
  v.voronoi
 
 
  there is some replacement test code to look at in grass-addons as well:
   http://trac.osgeo.org/grass/browser/grass-addons/vector/voronoi

 While we are on this topic, is there a way to get a weigthed voronoi
 diagram using grass ?

 The ability to rank a point to tune the area's influence would be great,
 for that purpose I've been using an arcgis'extension* but with grass it
 is not possible**. Is there a way to get a similar result ?

 *http://www.geog.unt.edu/~pdong/software.htm
 **http://osdir.com/ml/gis.grass.user/2004-04/msg00036.html

 Regards,
 MORREALE Jean Roc

What exactly is a 'weighted voronoi diagram' (the link does not work)? If you 
are after a weighted network connecting points, here is one approach:

http://casoilresource.lawr.ucdavis.edu/drupal/node/698

Dylan



-- 
Dylan Beaudette
Soil Resource Laboratory
http://casoilresource.lawr.ucdavis.edu/
University of California at Davis
530.754.7341
___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] Thiessen Polygons

[Dylan Beaudette]

On Wed, Feb 11, 2009 at 5:03 PM, Dylan Beaudette
debeaude...@ucdavis.edu wrote:
 On Wednesday 11 February 2009, MORREALE Jean Roc wrote:
 Hamish a écrit :
  Kurt Springs wrote:
  Does anyone know how to do Thiessen Polygons in GRASS?
 
  I need to use them around various types of megalithic tomb
  sites that are in point vector files.
 
  v.voronoi
 
 
  there is some replacement test code to look at in grass-addons as well:
   http://trac.osgeo.org/grass/browser/grass-addons/vector/voronoi

 While we are on this topic, is there a way to get a weigthed voronoi
 diagram using grass ?

 The ability to rank a point to tune the area's influence would be great,
 for that purpose I've been using an arcgis'extension* but with grass it
 is not possible**. Is there a way to get a similar result ?

 *http://www.geog.unt.edu/~pdong/software.htm
 **http://osdir.com/ml/gis.grass.user/2004-04/msg00036.html

 Regards,
 MORREALE Jean Roc

Strike that. Link appears to work and RTFM-ing the user guide...

Dylan
___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] Creating turbidity raster file

[Dylan Beaudette]

On Monday 09 February 2009, Hamish wrote:
 buraq wrote:
  I have formatted csv files for every month which is like
  latitude;longitude;turbidityValue. So I want to create
  raster files from these csvs for using in r.sun.

 v.in.ascii - convert .csv to a vector points file
 g.region  # set up raster grid
 v.surf.rast - interpolate vector points to raster surface


Minor correction:

v.surf.rast --- v.surf.rst




 I have worried about r.sun's Linke Turbidity factor values in areas with
 big changes in elevation. Is turbidity value heavily dependent on altitude?


It should, as it is based on a measure of optical thickness (air mass):

 Extracted from a document --
m = \frac{1}{sin(\alpha) + 0.15(\alpha + 3.885)^{-1.253}} e^{-0.0001184 \times 
A}

\noindent
where $\alpha$ is the solar elevation angle at (12:00 pm local time) on each 
day of the year, and $A$ is the weather station elevation in meters above 
mean sea level (MSL). 
 Extracted from a document --

This stuff is also on pages 40-41 of this document:
http://169.237.35.250/~dylan/d_e_beaudette-ms_thesis.pdf

Cheers,

Dylan


 Hamish







-- 
Dylan Beaudette
Soil Resource Laboratory
http://casoilresource.lawr.ucdavis.edu/
University of California at Davis
530.754.7341
___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] v.in.ascii

[Dylan Beaudette]

On Tuesday 03 February 2009, Harald Hofmann wrote:
 Hello all,
 I have been using GRASS now for a couple of years but I am new to the
 mailing list.
 Nevertheless, I could still not solve one problem to a satisfactory extend.
 What I mean is the header import in v.in.ascii.
 I have point data with chemical analysis in the attribute table. Every
 time I import a text file, containing these tables and using v.in.ascii,
 the header is not imported, which is quit annoying when you have 20 or
 30 different parameters.
 I have solved it in a, some kind of clunky, way. After importing the
 text file, I open the dbf in OOCalc and replace the first line with the
 header that I copy from the original text file.
 I was just wondering if there isn't an easier, more elegant method to do
 so.

 Thanks
 Harald

Which version of GRASS?

-- 
Dylan Beaudette
Soil Resource Laboratory
http://casoilresource.lawr.ucdavis.edu/
University of California at Davis
530.754.7341
___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] starspan with grass support compilation failure

[Dylan Beaudette]

On Thu, Jan 29, 2009 at 4:22 AM, Nikos Alexandris
nikos.alexand...@felis.uni-freiburg.de wrote:
 On Wed, 2009-01-28 at 18:05 +0100, Nikos Alexandris wrote:
 Not sure if I should post this here. I have a problem compiling starspan
 with support for GRASS.

 # configuring reports no problems
 ./configure --with-grass=/usr/local/grass-6.5.svn

 # attempt to compile...
 make

 [...]
 src/stats/Stats.cc:232: error: 'sort' was not declared in this scope
 make: *** [Stats.o] Error 1

 The same problem appears also with a bare ./configure.

 The same error appears with all of the three versions of starspan
 1.2.01, 1.2.02, 1.2.03.

 Perhaps it's not starspan-related.


Hmm. Check the archives for Jonathan Greenberg -- he is one of the
lead developers. It would be nice to get starspan into OSGEO so that
questions / code patches ect. can be handled in an efficient manner.
Starspan is an emmensly important GIS tool, but suffers from a lack of
developers, documentation, and bug handling.

Perhaps I should take these comments to the OSGEO mailing list... What
do you think Jonathan?

Cheers,

Dylan
___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] Aspect direction in r.slope.aspect

[Dylan Beaudette]

On Wednesday 21 January 2009, Glynn Clements wrote:
 Michael Barton wrote:
   I was just wondering why aspect is measured from the east and
   increased counterclockwise instead of the north and clockwise?  It
   seems to run counter to what is done in other GISs.
 
  For my 2 cents worth, it seems to make a lot more sense for a
  *geographic* information system for all output to follow the same
  spatial organizational standards. I understand the history of the east
  is 0 convention for parts of GRASS. And I also appreciate the
  importance of not breaking code-dependent features within versions.
  However, that does not mean that we should keep a non-standard way of
  measuring direction for select modules (like r.slope.aspect), while
  measuring from north for others, simply because the program started
  out that way. Version changes are a time when we can rethink and
  standardize different modules that have evolved independently. Scripts
  are likely to break across the 6/7 transition for other reasons anyway.
 
  That said, a functionally simple approach that would not be quite so
  disruptive would be to add a flag to switch to count from east mode
  (with count from north as the default) or even a flag to switch to
  count from north mode (with count from east being the default if
  backward compatibility is indeed very important in this case).

 I would suggest generalising it to scale and offset, so that you can
 use degrees, grad, radians, etc. E.g.:

   scale   offset  description
   360 0   degrees CCW from east
   -36090  degrees CW from north
   2*pi0   radians CCW from east
   -400100 grad CW from north

 We also need a flag to indicate whether angles are signed or unsigned,
 e.g. 0..360 vs -180..180. Off the top of my head:

   struct angles {
   double scale, offset;
   int is_signed;
   };

   double G_import_angle(const struct angles *f, double a)
   {
   return (a - f-offset) * (2*M_PI) / f-scale;
   }

   double G_export_angle(const struct angles *f, double a)
   {
   a = f-offset + a * f-scale / (2*M_PI);
   return a - f-scale * floor(a / f-scale + f-is_signed ? 0.5 : 
 0);
   }

 We would also want a function to set the conversion parameters from a
 string, so that common cases can be named (e.g. deg,geo for
 degrees/CW/north, rad,math for radians CW from east).

 Then, modules such as r.slope.aspect would get an angles= parameter,
 with the existing interpretation as the default.

If it wouldn't be too hard to implement, this seems like a nice generalized 
solution. I had forgotten about the output from r.param.scale...

Dylan

-- 
Dylan Beaudette
Soil Resource Laboratory
http://casoilresource.lawr.ucdavis.edu/
University of California at Davis
530.754.7341
___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] To change DEM resolution

[Dylan Beaudette]

On Monday 19 January 2009, Edmondo Elisei wrote:
 I've a DEM with 90m m res. (from CGIAR SRTM) and I need drape over it a map
 tif (resolution 1m).

 Seeing the resultant map in NViz the quality is very low. I suppose it
 depends from the different resolutions.
 There is a way to change the DEM resolution?

 Thanks

 Regards

 Edmondo

if you are using GRASS64, check out r.resamp.interp, otherwise r.bilinear 
should do the trick.

Cheers,

Dylan

-- 
Dylan Beaudette
Soil Resource Laboratory
http://casoilresource.lawr.ucdavis.edu/
University of California at Davis
530.754.7341
___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] interpolating with a covariate - v.vol.rst

[Dylan Beaudette]

On Tue, Jan 6, 2009 at 2:33 PM, Markus Neteler nete...@osgeo.org wrote:
 Dylan,

 On Tue, Jan 6, 2009 at 10:01 PM, Dylan Beaudette
 debeaude...@ucdavis.edu wrote:
 Hi,

 For some crazy reason I was under the impression that it is possible to do
 interpolation with a covariate with v.vol.rst. Are there any examples on how
 to parameterize this module, when a 2D surface is requested, rather than a 3D
 volume. I noticed the 'cellinp' argument for a cross-section, but this is not
 quite what I am after. I am looking to do something very similar to
 interpolation of rainfall data, taking into account the orographic effect of
 terrain.

 This was my main business (say, of our cluster) over the last months :)
 You can do that. I am using the elevation model as auxiliary variable:

 # something like this:
 v.vol.rst in=vectpoints cellinp=dem wcolumn=pointval cellout=rst2d

 cellout delivers the 2D map, extracted from the volume along the
 dem map.

 Hope this helps
 Markus


Thanks Markus. One more question: have you found a good compromise in
the 3D region settings- i.e. some ratio of horizontal:vertical
resolution that gives good results and doesn't take too long to
compute?

Cheers,

Dylan
___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

[GRASS-user] interpolating with a covariate - v.vol.rst

[Dylan Beaudette]

Hi,

For some crazy reason I was under the impression that it is possible to do 
interpolation with a covariate with v.vol.rst. Are there any examples on how 
to parameterize this module, when a 2D surface is requested, rather than a 3D 
volume. I noticed the 'cellinp' argument for a cross-section, but this is not 
quite what I am after. I am looking to do something very similar to 
interpolation of rainfall data, taking into account the orographic effect of 
terrain.

I would be happy to add an example or two to the man pages.

Cheers,

Dylan

-- 
Dylan Beaudette
Soil Resource Laboratory
http://casoilresource.lawr.ucdavis.edu/
University of California at Davis
530.754.7341
___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] r.quantile: what it exaclty do?

[Dylan Beaudette]

On Fri, Jan 2, 2009 at 4:55 AM, Jarek Jasiewicz jar...@amu.edu.pl wrote:
 Hi
 I tirend used r.quantile on SRTM data for Poland (min value about to 0, max
 value about to 2500 m a.s.l)

 r.quantile input=pol...@permanent quantiles=4
 percentiles=0.001,0.01,0,1,0.25,0.50,0.75,0.90,0.99,0.999 bins=100

 and I recived:

 0:0.00:0.00
 1:0.001000:0.00
 2:0.01:0.00
 3:0.25:0.00
 4:0.50:1.00
 5:0.75:2.00
 6:0.90:3.00
 7:0.99:3.00
 8:0.999000:3.00
 9:1.00:3.00

 It looks like the results are divided by 1000 and rounded to the nearest
 integer.

 Did I something wrong?

 Jarek


I thought that the user supplies one of [quantiles=] | [percentiles=]
... could that be related to the odd output. I have verified (with R)
that r.quantile can compute correct quantiles... however, on my system
I need to set bins=100 or so, as the higher i set 'bins' odd things
would happen.

Glynn should know for sure.

Dylan
___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] i.cluster and i.maxlik with grayscale image

[Dylan Beaudette]

On Thu, Jan 1, 2009 at 10:57 AM, Micha Silver mi...@arava.co.il wrote:


 One set of aerial photos are from the 1950's -1960's. These are all black
 and white photos: single band tiff.  The newer set of aerial photos are
 color images from 2005-2007.
 The region is a savanna area with sparse trees and bushes. We need to
 compare how many trees, and their size, 50 years ago with the situation
 today.

 Any additional pointers are most welcome. Thanks,
 Micha

Hi Micha,

i.smap has worked well for our studies in Oak Woodland ecosystems.
Here is an example using RGB imagery:

http://casoilresource.lawr.ucdavis.edu/drupal/node/548

Dylan
___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user