Re: [GRASS-user] Local Relief Model tool
Eric: To install in a linux environment, I followed the instructions at http://grasswiki.osgeo.org/wiki/GRASS_and_Python#Testing_and_installing_Python_extensions. I haven't been able to find solid information on how users who don't compile themselves can install an addon that isn't available via g.extension. In general it's quite simple, users maintain a directory with all of their executable scripts in it and before starting GRASS add export GRASS_ADDON_PATH=/path/to/files to their ~/.bashrc. (~/.grass.bashrc is no good, the variable has to be set before grass starts) Then it magically finds them. The g.extension module(s) just fits itself into that and creates you an addon dir if one wasn't already set. For scripts there is no other install or compiling needed, just put it in a dir somewhere which is in the $PATH. Python might be a problem, but if you just call your module by its full name it should be ok (so with or without .py, just be sure to match the exact filename). There is http://grasswiki.osgeo.org/wiki/Compile_and_Install#Scripts, but I'm not sure how that applies to Windows users. Any advice in this area would be much appreciated. The GRASS 6 make system is still missing build support for python scripts (often it tries to reuse the shell Script.make, which mostly works on Linux). Smooth building with correct .bat file wrappers on Windows remains an issue. It can be done, I've seen it work, but still needs a new PythonScript.make to work smoothly. (similarly user- created personal shell scripts for GRASS 7 should have a ShellScript.make to help folks who need that, even if there are none in the main release.) regards, Hamish ___ grass-user mailing list grass-user@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-user
Re: [GRASS-user] v.in.ascii : ERROR: write sidx
Le Jeune Yann wrote:
Hello,
I have just installed GRASS 7.0 svn 57844 on Linux Mint 15.
v.in.ascii stopped during the build process : ERROR: write sidx: wrong node
position in file
Like this :
Scanning input for column types...
Number of columns: 3
Importing points...
100%
Building topology for vector map PTS_menhir2_pc_88M@menhir2...
Registering primitives...
88509169 primitives registered
88509169 vertices registered
Building areas...
100%
0 areas built
0 isles built
Attaching islands...
Attaching centroids...
100%
Number of nodes: 0
Number of primitives: 88509169
Number of points: 88509169
Number of lines: 0
Number of boundaries: 0
Number of centroids: 0
Number of areas: 0
Number of isles: 0
ERROR: write sidx: wrong node position in file
If you are using a 32bit OS, you might need large file support (LFS)
which is by default enabled. LFS can but should not be disabled with
--disable-largefile.
Do you have enough free disk space?
If you do not need topology for the points, you can try v.in.ascii -b.
The command was :
v.in.ascii -z input=${FILE} output=${NAME_OUT} separator=${FS} z=3
--overwrite
From a shell script.
Is this a known issue ?
No.
Markus M
___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user
Re: [GRASS-user] Grass SQLite driver math funtions
Dear list, dear Markus, 2013/9/26 Markus Neteler nete...@osgeo.org On Wed, Sep 25, 2013 at 11:42 AM, Enrico Gallo enrico.ga...@gmail.com wrote: Dear list, I am struggling with the use of simple math functions in SQL expression, using SQLIte and db.execute. I am writing a script for multiplatform end-users, so nor compiling SQLite math contrib library nor piping data to bc using command line seem to me feasable solutions. Do you have any suggestion? Could you post an example? GRASS 6.4 from North Carolina data set sqlite mapset g.copy vect=railroads@PERMANENT,foo v.db.addcol map=foo columns=power double precision,distance double precision,level double precision v.db.update map=foo column=power value=80 v.db.update map=foo column=distance value=100 v.db.update map=foo column=level value=power-distance # OK v.db.update map=foo column=level value=power-log10(distance) DBMI-SQLite driver error: Error in sqlite3_prepare(): no such function: log10 ERROR: Error while executing: 'UPDATE foo SET level=power-log10(distance) ' same error for sqrt(), log(), etc see also [1] (in Italian) with dbf driver; SQLite suggestion seems wrong Including SQLite math functions in the standard binary GRASS GIS distribuition could be a long term solution? I think this is the choice SpatialLite did since 2.3 version. You mean http://www.gaia-gis.it/gaia-sins/spatialite-sql-3.0.0.html#math hence http://www.sqlite.org/contrib -- extension-functions.c (50.96 KB) contributed by Liam Healy on 2010-02-06 15:45:07 Provide mathematical and string extension functions for SQL queries using the loadable extensions mechanism. Math: acos, asin, atan, atn2, atan2, acosh, asinh, atanh, difference, degrees, radians, cos, sin, tan, cot, cosh, sinh, tanh, coth, exp, log, log10, power, sign, sqrt, square, ceil, floor, pi. String: replicate, charindex, leftstr, rightstr, ltrim, rtrim, trim, replace, reverse, proper, padl, padr, padc, strfilter. Aggregate: stdev, variance, mode, median, lower_quartile, upper_quartile. If you refer to this file, then it is more related to (your) SQLite installation rather than GRASS itself since GRASS just calls SQLite. best, Markus It's exactly what I meant So there is not a way to use this kind of funtions with vector attributes without compile extra library and load it in SQLIte? Any python workaround? Users need to change database, switching to MySQL or PostgreSQL? Many thanks, Enrico [1] http://listserv.unipr.it/pipermail/grass-italia/2012-April/005897.html ___ grass-user mailing list grass-user@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-user
Re: [GRASS-user] Local Relief Model tool
On Thu, Sep 26, 2013 at 12:54 AM, Hamish hamis...@yahoo.com wrote:
Vaclav wrote:
Hi Eric,
actually I was working on the same module (named r.local.relief), so
we have some duplication now. It is also based on [Hesse2010]. I was
about to commit it to GRASS Addons but I need to write documentation
first. For now, I'm adding the Python script into a attachment.
I've quickly tested both and they give slightly different results. The
other visible difference is the contours-to-really_smooth_elevation
step.
Hi,
could you discuss the advantage of the local relief method? I take it that
the idea is to remove the background signal to highlight local detail?
v.surf.rst or v.surf.bspline with a really big search window and loose
tension might also be good for a smoothed backgroundto subtract away, and
see also recent discussion about trying to get planar trend surfaces out of
r.cog addon on the grass-dev ML (a work in progress).
Should fine relief be removed as well? (so gate filter and not just a low
or high pass one)
It seems to me that the contouring step is needlessly lossy and artifact
prone, and that you might get better results using the r.surf.contour
module to recreate a raster from the contour lines. Or perhaps better use a
{v,r}.random sampling technique as input to one of the v.surf spline
modules or r.surf.nnbathy -- less artifacts than interpolating from contour
lines. Also note that the contour method will flatten off the tops of
features which are smaller than your contour step level. (so choice of
contour step size becomes very important)
The thought behind the contouring step is that when you apply a low pass
filter and subtract it from the DEM, the large scale features (based on the
selected kernel size) are eliminated and the small-scale features are
smoothed, resulting in bias toward small features and an underestimation of
the magnitude of the local relief elevations as the spatial extent of the
feature increases. By getting the 0-meter contours, converting them to
points, and getting the elevations of the original DEM for those points,
you're basically able to cut the large-scale features out of the original
DEM, leaving the local relief behind without smoothing.
See also http://grasswiki.osgeo.org/wiki/Contour_lines_to_DEM
for an analysis and comparison of interpolation methods for creating
surfaces from contour lines. (spoiler: the designed for the task
r.surf.contour wins)
regards,
Hamish
___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user
Re: [GRASS-user] Grass SQLite driver math funtions
On Thu, Sep 26, 2013 at 2:18 PM, Enrico Gallo enrico.ga...@gmail.com wrote:
So there is not a way to use this kind of funtions with vector attributes
without compile extra library and load it in SQLIte?
Any python workaround?
this example works only in grass7, otherwise you have to write your
python code to work directly with the sqlite table.
{{{
from grass.pygrass.modules.shortcuts import general as g
from grass.pygrass.vector import VectorTopo
# copy the vector map
g.copy(vect=['railroads', 'foo'], overwrite=True)
# define the column
cols = [('power', 'double precision'),
('distance', 'double precision'),
('level', 'double precision')]
# instantiate the vector map
foo = VectorTopo('foo')
foo.open('rw')
# add the new columns
for cname, ctype in cols:
foo.table.columns.add(cname, ctype)
# add some values
foo.table.execute('UPDATE foo SET power=80')
foo.table.execute('UPDATE foo SET distance=100')
foo.table.conn.commit()
#
# work around
import math
for geo in foo:
geo.attrs['level'] = geo.attrs['power'] - math.log10(geo.attrs['distance'])
}}}
Note that this is far to be efficient...
If you compile sqlite including the support for the extra functions,
you simply need to substitute the workaroud with:
{{{
foo.table.execute('UPDATE foo SET level=power - log10(distance)')
foo.table.conn.commit()
}}}
Best regards
Pietro
___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user
Re: [GRASS-user] v.in.ascii : ERROR: write sidx
If you are using a 32bit OS, you might need large file support (LFS) which is by default enabled. LFS can but should not be disabled with --disable-largefile. i use a 64 bit system, with LFS Do you have enough free disk space? Yes If you do not need topology for the points, you can try v.in.ascii -b. that works for me, Thanks a lot !! ++ yann ___ grass-user mailing list grass-user@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-user
Re: [GRASS-user] v.in.ascii : ERROR: write sidx
Sorry, in Fact i still need topology for further processes, Any ideas ? Thanks yann 2013/9/26 Le Jeune Yann lj.y...@gmail.com If you are using a 32bit OS, you might need large file support (LFS) which is by default enabled. LFS can but should not be disabled with --disable-largefile. i use a 64 bit system, with LFS Do you have enough free disk space? Yes If you do not need topology for the points, you can try v.in.ascii -b. that works for me, Thanks a lot !! ++ yann -- Yann Le Jeune Web : http://www.paleosystem.fr/ylj_pro/ ___ grass-user mailing list grass-user@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-user
Re: [GRASS-user] v.in.ascii : ERROR: write sidx
On Thu, Sep 26, 2013 at 7:05 PM, Le Jeune Yann lj.y...@gmail.com wrote: Sorry, in Fact i still need topology for further processes, Any ideas ? May be now you are able to build the topology using v.build module, or do you have still a error? Pietro ___ grass-user mailing list grass-user@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-user
Re: [GRASS-user] v.in.ascii : ERROR: write sidx
I tried v.build after v.in.ascii -b and there's the same error at the same step.. :( 2013/9/26 Pietro peter.z...@gmail.com On Thu, Sep 26, 2013 at 7:05 PM, Le Jeune Yann lj.y...@gmail.com wrote: Sorry, in Fact i still need topology for further processes, Any ideas ? May be now you are able to build the topology using v.build module, or do you have still a error? Pietro -- Yann Le Jeune Web : http://www.paleosystem.fr/ylj_pro/ ___ grass-user mailing list grass-user@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-user
Re: [GRASS-user] v.in.ascii : ERROR: write sidx
Yann wrote:
The command was :
v.in.ascii -z input=${FILE} output=${NAME_OUT} separator=${FS} z=3
--overwrite
Hi,
just to note -- the {curly} brackets do nothing to
protect from spaces in filenames in this situation,
double quotes should be used for that.
the curly brackets protect the variable name, not its
contents, so are mostly useful for when a letter,
number, or _ follows the variable name.
I actually think it is dangerous to use the curly
brackets because it tricks people into thinking they
are protected when they are not.
regards,
Hamish
___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user
Re: [GRASS-user] v.in.ascii : ERROR: write sidx
Hi,
Thanks for the {curly} thing !
I tried v.in.ascii from the Layer manager (i use sqlite driver) :
(Thu Sep 26 23:58:29
2013)
v.in.ascii -z --overwrite --verbose
input=/media/yann/DATA_SIG/3D_photo/menhir2_pc_88M.csv
output=PTS_menhir2_pc_88M separator= z=3
WARNING: Vector map PTS_menhir2_pc_88M already exists and will be
overwritten
Using native format
Scanning input for column types...
Maximum input row length: 30
Number of columns: 3
Importing points...
Building topology for vector map PTS_menhir2_pc_88M@menhir2...
Registering primitives...
88509169 primitives registered
88509169 vertices registered
Building areas...
0 areas built
0 isles built
Attaching islands...
Attaching centroids...
Topology was built
Number of nodes: 0
Number of primitives: 88509169
Number of points: 88509169
Number of lines: 0
Number of boundaries: 0
Number of centroids: 0
Number of areas: 0
Number of isles: 0
ERROR: write sidx: wrong node position in file
(Fri Sep 27 00:44:28 2013) Command finished (45 min 59 sec)
I have some disk space, but i looked at my system monitor and my RAM went
to SWAP (i have only 16 Go RAM, 15 Go SWAP) The swap don't seems to have
been totally used (but at least half) and i have not in front of my PC when
the error occurred.
I will try this again with more RAM.
Thanks for your answers.
yann
2013/9/26 Hamish hamis...@yahoo.com
Yann wrote:
The command was :
v.in.ascii -z input=${FILE} output=${NAME_OUT} separator=${FS} z=3
--overwrite
Hi,
just to note -- the {curly} brackets do nothing to
protect from spaces in filenames in this situation,
double quotes should be used for that.
the curly brackets protect the variable name, not its
contents, so are mostly useful for when a letter,
number, or _ follows the variable name.
I actually think it is dangerous to use the curly
brackets because it tricks people into thinking they
are protected when they are not.
regards,
Hamish
___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user
Re: [GRASS-user] v.in.ascii : ERROR: write sidx
Yann wrote: I tried v.in.ascii from the Layer manager (i use sqlite driver) : (Thu Sep 26 23:58:29 2013) v.in.ascii -z --overwrite --verbose input=/media/yann/DATA_SIG/3D_photo/menhir2_pc_88M.csv output=PTS_menhir2_pc_88M separator= z=3 WARNING: Vector map PTS_menhir2_pc_88M already exists and will be overwritten Using native format Scanning input for column types... Maximum input row length: 30 Number of columns: 3 Importing points... Building topology for vector map PTS_menhir2_pc_88M@menhir2... Registering primitives... 88509169 primitives registered 88509169 vertices registered Building areas... 0 areas built 0 isles built Attaching islands... Attaching centroids... Topology was built Number of nodes: 0 Number of primitives: 88509169 Number of points: 88509169 Number of lines: 0 Number of boundaries: 0 Number of centroids: 0 Number of areas: 0 Number of isles: 0 ERROR: write sidx: wrong node position in file (Fri Sep 27 00:44:28 2013) Command finished (45 min 59 sec) I have some disk space, but i looked at my system monitor and my RAM went to SWAP (i have only 16 Go RAM, 15 Go SWAP) The swap don't seems to have been totally used (but at least half) and i have not in front of my PC when the error occurred. I will try this again with more RAM. Ok, 88 million points and running 64bit Linux. Since 3D points and no additional data columns columns I think it will default to not creating a database, so you don't have to worry about the -t flag or the memory needed for that, just the finite amount of RAM per point for topology. Maybe 64GB RAM will be enough for 88M, perhaps more, but it will need a lot to build topology for all those points. May I ask what process you'd like to do with the data? Perhaps r.in.xyz + r.to.vect helps reduce the number of points to something more manageable. The v.in.ascii '-r' flag can be helpful for importing only those points within the current computational region of interest, and if importing without building topology (the '-b' flag you tried before) many of the LiDAR modules and v.surf.rst have been modified to work without needing it. If it is some other vector module maybe we can modify it to load data without topology too. Is it sparse x,y data or gridded? regards, Hamish ___ grass-user mailing list grass-user@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-user
