[GRASS-dev] r.out.gdal, again
Hi, I have a wishlist for r.out.gdal together with a new version (link below), and would like to put it up for discussion: Changes in my version are: Colortable export A new flag -c is provided to export a colortable, default to no. Colortables in GeoTIFF files are sometimes not properly rendered by other GIS applications and the file might then be displayed all black, even when the raster values are correct. The same GeoTIFF file without colortable is displayed ok. Requested colortable export only works when there is an entry in directories colr or colr2. GRASS has no control over how a colortable is written to the exported file, this is handled by GDAL and dependent on the file format. GRASS uses intelligent color rules, whereas e.g. GeoTIFF needs a color table, i.e. one entry for each potential, not actual data value. Nodata value handling If a nodata value was supplied, this value is tested whether it is within the range of the selected data type and adjusted if necessary. Specifying e.g. a nodata value of - and using Byte as data type (range 0 - 255) is no longer accepted, it will be adjusted to 0. Similarly, a nodata value of would be adjusted to 255 for Byte type. More generally, if a supplied nodata value is smaller than the supported minimum, it is adjusted to the supported minimum, and if a supplied nodata value is larger than the supported maximum, it is adjusted to the supported maximum. Current region extends aligned to raster resolution The old r.out.gdal (GRASS-6.2.x) worked with the extends and resolution of the raster to be exported. The new r.out.gdal C-module works with the extends and resolution of the current region. My version works with the extends of the current region and aligns them to the input raster map resolution to avoid undesired implicit resampling if the resolution of the current region does not match the input raster resolution. If a raster map layer should be exported in a different resolution the raster must now be resampled prior to exporting. Raster band statistics Raster band statistics (min, max, mean, stddev) are written to the exported file if supported, GDAL decides. Since r.out.gdal uses current region extends, these statistics are calculated for the current region extends, replacing NULLs with the nodata value. I'm not sure if it would be better to exclude NULLs for raster band statistics. These statistics can be used by other GIS applications to get the actual data range (an argument for inclusion of nodata values in raster stats), but not many GIS applications actually make use of this information. ESRI products e.g. initially report the potential data range, not the real data range. This is usually not an issue with Byte type because of its small range, but with e.g. Int32 or Float32, ESRI products often initially display an all black image (not only GeoTIFF, also e.g. ESRI .hdr labelled files). Adjusting the display mode (Symbology) helps. Data type and range test If the range of the raster to be exported is not covered by the selected data type, r.out.gdal is aborted with an error message giving information on the range of the raster to be exported as well as the range of GDAL data types. The user can then easily compare the raster data range with the ranges of GDAL data types and select an appropriate data type. Updated documentation Documentation is updated, amongst other minor changes with ranges of GDAL data types and recommended settings for best allround compatibility of GeoTIFF files. The recommended settings are based on discussions in GRASS mailing lists, GRASS ticket 73, GDAL documentation and own testing. Apart from colortable export defaulting to no, these recommended settings are not enforced or set as default. Maciek looked into GeoTIFF optimisation, e.g. adding overviews, tiles vs. strips, strip dimensions, internal compression methods. Since GeoTIFF is so widely used, this information could also be included in the r.out.gdal documentation. This version of r.out.gdal was compiled with grass-6.4.svn_src_snapshot_2008_08_16 and gdal-1.5.2. Source code has been formatted using tools/grass-indent.sh I tested with QGIS-0.9.1 on Linux 64bit, QGIS -0.11.0-1 on XP, SAGA-2.0.3 on XP, gvSIG-1.1.1 (32bit binary) on Linux 64bit, gvSIG-1.1.1 on XP, PCI Geomatica FreeView 9.1 on XP, ER Viewer 7.2 (ER Mapper) on XP, and ArcMap 9.2 (ESRI) on XP. It is not so easy to get hold of GIS applications, also commercial, that do not use gdal... Tests were performed with the elevation raster map layer in the North Carolina sample dataset, creating a MASK for elevation values = 70m, needed for testing of nodata handling. The elevation raster was exported as Byte, UInt16, Int32, and Float32, once with colortable (Byte and UInt16 only, colortable export for other data types not supported by gdal), once without colortable, always as GeoTIFF. Test results: QGIS-0.9.1 on Linux 64bit display: all ok colortable: all ok cell values: all ok
Re: [GRASS-dev] r.out.gdal, again
Maciej Sieczka wrote: Markus Metz pisze: Nodata value handling If a nodata value was supplied, this value is tested whether it is within the range of the selected data type and adjusted if necessary. Specifying e.g. a nodata value of - and using Byte as data type (range 0 - 255) is no longer accepted, it will be adjusted to 0. Similarly, a nodata value of would be adjusted to 255 for Byte type. IMHO r.out.gdal should abort with an error rather than silently adjust user-specified values. Aborting with an error is definitively an option, but then the original behaviour of r.out.gdal should be changed too. The original r.out.gdal chooses a nodata value according to the selected output data type if none was specified, without message or warning. Later on (original nodata value handling), a warning message is issued if there are NULLs in the raster to be exported, irrespective of whether a nodata value was specified by the user or not. With my version, the nodata value is not silently adjusted, a warning message is issued. I wanted to keep the behaviour similar to the original version, and if everything is all right, there is no difference apart from the additional -c flag and a warning if colortable export takes place. r.out.gdal nodata handling could be changed so that the user has to provide a nodata value if there are NULLs in the raster, otherwise r.out.gdal aborts with an error. Further on, r.out.gdal can also abort with an error if the specified nodata value is not covered by the range of the selected data type, instead of adjusting the nodata value and issuing a warning message. IMHO, either a warning or an error message should be issued and the type of message - warning or error - should be consistent for nodata handling in the whole module. If the general policy is not to adjust user-specified values, even with a warning message, but to abort with an error and provide information on what's wrong and how to fix it, then it should be changed to aborting with an error. In any case, this is easy to change. Markus ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] r.out.gdal, again
Dylan Beaudette wrote: On Wednesday 20 August 2008, Glynn Clements wrote: Markus Metz wrote: r.out.gdal nodata handling could be changed so that the user has to provide a nodata value if there are NULLs in the raster, otherwise r.out.gdal aborts with an error. That's reasonable for integer maps, but for FP maps it makes more sense to simply default to NaN as the no-data value (the two are essentially the same concept, so there is no information loss in using NaN). Do other software packages understand 'Nan' ? Do other file formats understand Nan? Reading a bit in GDAL API documentation lets me suspect that NaN is not allowed, because for GeoTIFF, you can specify a NoData value as in the current implementation or you can add an internal mask, no mentioning of writing NaN for Float32 or Float64 to a GDAL raster band. Did I overlook something? Markus ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
[GRASS-dev] Re: big region r.watershed
Hamish wrote: Markus Metz wrote: The original version uses very little memory, so assuming that GRASS runs today on systems where at least 500MB RAM are available I changed the parameters for the seg mode, more data are kept in memory, speeding up the seg mode. Looking at other modules using the segment library (e.g. v.surf.contour, r.cost), it seems that there is not one universally used setting, instead the segment parameters are tuned to each module. The new settings work for me, but not necessarily for others, and maybe using 500MB is a bit much. fwiw r.terraflow has a memory= option, the default is 300mb. AFAIU, the bigger you make that, the smaller the on-disk temp files need to be (ie work-around to keep tmp files 2gb for 32bit filesystems). a number of modules like r.in.poly have a rows= option, which I didn't really understand until I got into the code. (hold at most that many region rows (all columns) in memory at once). Interestingly the default value has scaled quite well over the years. and other modules like r.in.xyz have percent= (0-100) for how much of the map to keep in memory at once. A default value that scales well over the years would be preferable, but performance of r.watershed.fast -m is really poor if whole columns (or rows ) are kept in memory and much better if segments have equal dimensions. Interestingly, segments of 200 rows and 200 columns are processed fastest, faster than e.g. 150 rows and columns or 250 rows and columns. The more segments are kept in memory the better. Right now I don't want to introduce a new option to give the user control over how much memory is used (be it MB memory, number of rows or percent of the map) because I want to keep all options of r.watershed.fast identical to the original version. I'm still not happy with the speed of the segmented version of r.watershed.fast, but at least it is magnitudes faster than the in-memory version of the original r.watershed. Maybe the iostream library that came with r.terraflow can be used for r.waterhed -m as well. Markus ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
[GRASS-dev] r.watershed.fast in GRASS-Addons svn repository
Hi all, r.watershed.fast is now in the GRASS AddOns svn repository and I added an entry in the GRASS AddOns wiki page. I hope everything went all right and is at the proper place: in the svn repository I made a new directory r.watershed.fast in directory raster and the new entry in the GRASS AddOns wiki page is under 5.GRASS 6.x - 5.2 Raster add-ons - 5.2.32 r.watershed.fast. If I should change something (text or location) please let me know! Markus ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] r.watershed.fast in GRASS-Addons svn repository
As requested in the psc mailing list, I have renamed the new version to r.watershed2 both in the svn repository and the GRASS wiki. I have read the request for renaming only now, sorry for the confusion! Markus Markus Metz wrote: Hi all, r.watershed.fast is now in the GRASS AddOns svn repository and I added an entry in the GRASS AddOns wiki page. I hope everything went all right and is at the proper place: in the svn repository I made a new directory r.watershed.fast in directory raster and the new entry in the GRASS AddOns wiki page is under 5.GRASS 6.x - 5.2 Raster add-ons - 5.2.32 r.watershed.fast. If I should change something (text or location) please let me know! Markus ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
[GRASS-dev] Re: [GRASS GIS] #73: r.out.gdal tiff output does not work
GRASS GIS wrote: #73: r.out.gdal tiff output does not work --+- Reporter: helena | Owner: grass-dev@lists.osgeo.org Type: defect | Status: new Priority: critical | Milestone: 6.4.0 Component: Raster | Version: svn-trunk Resolution: |Keywords: r.out.gdal, tiff Platform: Unspecified | Cpu: Unspecified --+- Comment (by neteler): Markus (Metz), what about integrating your fixes from http://markus.metz.giswork.googlepages.com/r.out.gdal.conservative.tar.gz ? Markus Sure, no objections from my side, I'm using this version only. But r.out.gdal is a very important module of GRASS and maybe some more testing is required. Also, the new features may not find approval by all, e.g. I've put in rather restrictive NULL cell handling: the user has to specify a nodata value that falls within the range of the selected datatype (Byte, UInt16, Int16, ...), a nodata value is no longer chosen automatically as in the original version. If a nodata value is not given, but NULL cells are present, r.out.gdal aborts with an error message. My version also no longer uses the current region resolution, instead the current region extends are aligned to the resolution of the raster to be exported to avoid any implicit resampling. And the colortable is only exported on request, and then with a warning message. If these changes are ok with you then integrate the changes, otherwise maybe only some, but not all changes could be integrated. More discussion needed on how to change/improve r.out.gdal? Markus ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] testing results of r.watershed2 against old r.watershed
It seems that I am at least in part responsible for the confusion about svn versions of r.watershed2/r.watershed.fast, sorry for that! The initial request was to make the changes to r.watershed available as a separate module in the grass-addons svn repository, and I created a new directory r.watershed2 there without any svn reference/information to r.watershed. The code was formatted with tools/grass_indent.sh so there shouldn't be any funny whitespaces/tabs. Now that the changes have been applied to grass-6.4 r.watershed2 should be removed from grass-addons, it's only confusing users. Actually I wanted to apply the changes of the r.watershed version in grass-7 to r.watershed2, especially naming of options without points and uppercase, but didn't get yet to it. Now that some changes have been applied to r.watershed, maybe this is sparking some interest in improving some parts here and there, as suggested by Helena with regard to lsfac and meter to foot conversion, also the suggestion of Isaac Ullah to apply a colortable to flow accumulation that is equivalent to the visual output and remove the visual option. Further on, as Isaac mentioned it remains the responsibility of the user to create a MASK excluding NULL cells in the input DEM before running r.watershed. I think that NULL cells should be always ignored and additionally any existing MASK respected. These changes shouldn't be too difficult to apply. If I suggest changes to the code again, I will supply diffs in the hope to support svn change tracking and to avoid the confusion caused by adding a module that appears new as far as svn is concerned. Markus Metz Hamish wrote: Hi, I did a bit more syncing between what is now in devbr6 for r.watershed1 and 2. a minimized diff for review can be found here: https://trac.osgeo.org/grass/attachment/ticket/344/r.w2.diff after review, that patch should be applied to r.watershed/ in devbr6 and trunk, and r.watershed2/ removed from devbr6. Hamish ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] testing results of r.watershed2 against old r.watershed
Hamish wrote:
Markus Metz wrote:
Actually I wanted to apply the changes of the r.watershed version in
grass-7 to r.watershed2, especially naming of options without points
and uppercase, but didn't get yet to it.
done. ('svn merge' did 99% of the work after devbr6 was solved)
Change option names also for r.watershed.ram and r.watershed.seg? There
are some options in uppercase.
see the man page for an example of making a nicely colored accum map
based on standard deviations.
if visual= is to be removed, that should only happen in grass 7.
Why not setting colors for accum in the module?
in my tests r.watershed(2) is 80 times faster than r.watershed(1). nice!
(35min - 25sec for RAM mode to complete; new SEG mode took 4.5min)
all maps appear the same as old r.watershed.
The speed increase is not static, the new version will be faster the
larger the region (more cells). For somewhat larger regions, the new
module is 1000 times faster.
#spearfish
g.region rast=elevation.10m
time r.watershed -m mem=800 elev=elevation.10m threshold=3000 \
accum=rw2_elev10m.acc \
drain=rw2_elev10m.drain \
basin=rw2_elev10m.basin \
str=rw2_elev10m.stream \
half.b=rw2_elev10m.halfb \
vis=rw2_elev10m.viz \
length.sl=rw2_elev10m.ls \
slope.st=rw2_elev10m.sls
One thing with -m (seg mode).. without -m (ram mode) it takes 166mb RAM.
With -m it took just under what I set memory= to. If I set mem=950 it
used 911mb RAM. Does it not know it only needs ~166mb instead of full
alloc?
It should, but I made a mistake in adjusting the number of open
segments. Please apply the diff attached.
Also, if I set -m memory=1200 the r.watershed.seg exits with an error (11):
[same elevation.10m which takes 166mb in RAM mode]
SECTION 1 beginning: Initiating Variables. 6 sections total.
D1/1: segs MB: 1200
D1/1: seg cols: 200
D1/1: seg rows: 200
D1/1:row segments: 7
D1/1: column segments: 10
D1/1: total segments: 70
D1/1: open segments: 419
D1/1: open segments after adjusting: 419
open segments after adjusting should be 70, fixed with diff.
SECTION 1b (of 6): Determining Offmap Flow.
100%
100%
WARNING: category information for [...] missing
[...]
G63 echo $?
11
top reported 1150mb allocated, and I have 2gb so plenty still to spare
and not Linux tearing down a runaway proc AFAICT..
so why the early exit?
That may be related to the number of open segments exceeding the number
of existing segments. After the changes as in the diff I could not
reproduce that error. Top now reports about 110 MB allocated for
segmented mode, no matter what I specified with memory.
There was also another error in init_vars.c, no memory allocated to char
*mb_opt which could cause a segfault, fixed with diff.
I wonder how many more bugs will surface after more testing...
Markus Metz
--- r.watershed2.orig/seg/init_vars.c 2008-11-29 11:45:15.0 +0100
+++ r.watershed2/seg/init_vars.c 2008-12-01 16:06:16.0 +0100
@@ -11,7 +11,6 @@
int fd, num_cseg_total, num_open_segs;
int seg_rows, seg_cols;
double segs_mb;
-char *mb_opt;
/* int page_block, num_cseg; */
int max_bytes;
@@ -30,6 +29,7 @@
/* dep_slope = 0.0; */
max_bytes = 0;
sides = 8;
+segs_mb = 300;
for (r = 1; r argc; r++) {
if (sscanf(argv[r], el=%[^\n], ele_name) == 1)
ele_flag++;
@@ -61,11 +61,7 @@
ls_flag++;
else if (sscanf(argv[r], ob=%[^\n], ob_name) == 1)
ob_flag++;
- else if (sscanf(argv[r], mb=%[^\n], mb_opt) == 1) {
- if (sscanf(mb_opt, %lf, segs_mb) == 0) {
- segs_mb = 300;
- }
- }
+ else if (sscanf(argv[r], mb=%lf, segs_mb) == 1) ;
else if (sscanf(argv[r], r=%[^\n], ril_name) == 1) {
if (sscanf(ril_name, %lf, ril_value) == 0) {
ril_value = -1.0;
@@ -155,8 +151,10 @@
num_open_segs = segs_mb / 2.86;
G_debug(1, segs MB: %.0f, segs_mb);
-G_debug(1, seg cols: %d, seg_cols);
+G_debug(1, region rows: %d, nrows);
G_debug(1, seg rows: %d, seg_rows);
+G_debug(1, region cols: %d, ncols);
+G_debug(1, seg cols: %d, seg_cols);
num_cseg_total = nrows / SROW + 1;
G_debug(1,row segments:\t%d, num_cseg_total);
@@ -169,8 +167,8 @@
G_debug(1, open segments:\t%d, num_open_segs);
/* nonsense to have more segments open than exist */
-if (num_open_segs nrows)
- num_open_segs = nrows;
+if (num_open_segs num_cseg_total)
+ num_open_segs = num_cseg_total;
G_debug(1, open segments after adjusting:\t%d, num_open_segs);
cseg_open(alt, seg_rows, seg_cols, num_open_segs);
___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] testing results of r.watershed2 against old r.watershed
Hamish wrote:
Hamish:
see the man page for an example of making a nicely colored accum map
based on standard deviations.
MMetz:
Why not setting colors for accum in the module?
If you like, but a simple linear color model will not work well:
Hmm, it does work with the existing visual output:
1. all negative values are set to 0
2. standard rainbow color rules are applied to the range 1 to 120,
leaving out anything beyond.
Since the bulk of the flow accumulation output has little flow, a simple
fixed linear model would work in most cases as it does for the visual output
Suggested color rules for flow accumulation including negative values
(offmap inflow), inspired from the color model of the visual output
covering the full range of flow accumulation values:
0% black
-200 black
-150 blue
-40 cyan
-20 green
0 yellow
20 green
40 cyan
150 blue
200 black
100% black
first making sure that 0% is smaller than -200 and 100% is larger than 200
The attached diffs for ram and seg mode apply these color rules.
Both visual output and these color rules may not look nice for DEMs with
very high resolution, e.g. LIDAR (just a guess that there will be more
cells with high accum values), then custom color rules can be set. A
fancier color model could use quartiles or quantiles, e.g. 20% steps.
The speed increase is not static, the new version will be
faster the larger the region (more cells). For somewhat
larger regions, the new module is 1000 times faster.
ok, can you suggest better wording for the release announcement?
Something like:
The new version of r.watershed produces results identical to the
original version at a substantial speed increase by optimizing the A *
search method. The speed increase with the new version is relative to
the size of the region, i.e. the more cells have to be processed the
higher is the speed gain. E.g a region with 2,000,000 cells is processed
about 100 times faster and a region with 20,000,000 cells is processed
about 5000 times faster (exact differences are system-dependent). More
technically, the time needed with the new version for a given region is
(in theory) proportional to the logarithm of the time needed with the
original version. [Now that is terrible wording...]
t1000 may as well be infinite, as before the user hit ^C after 2-3 days
and so it never finished. So this opens the door to analyzing much bigger
(ie modern) datasets; r.terraflow is nice for those, but doesn't provide
the catchment basin output maps.
In my personal opinion, flow accumulation of r.watershed is also more
realistic than flow accumulation of r.terraflow (SFD), but I have
admittedly not tested it in detail.
Regards,
Markus
--- r.watershed2.orig/seg/close_maps.c 2008-11-29 11:45:15.0 +0100
+++ r.watershed2/seg/close_maps.c 2008-12-02 11:11:56.0 +0100
@@ -7,11 +7,51 @@
int r, c;
CELL is_swale, value;
double dvalue;
+struct Range accRange;
+CELL min, max;
dseg_close(slp);
cseg_close(alt);
-if (wat_flag)
+if (wat_flag) {
cseg_write_cellfile(wat, wat_name);
+
+ /*
+ color rules for flow accumulation
+0% black
+ -200 black
+ -150 blue
+ -40 cyan
+ -20 green
+ 0 yellow
+20 green
+40 cyan
+ 150 blue
+ 200 black
+ 100% black
+ */
+
+ G_read_range(wat_name, this_mapset, accRange);
+ min = max = 0;
+ G_get_range_min_max(accRange, min, max);
+
+ G_init_colors(colors);
+ if (min -200)
+ G_add_color_rule(min, 0, 0, 0, -200, 0, 0, 0, colors);
+
+ G_add_color_rule(-200, 0, 0, 0, -150, 0, 0, 255, colors);
+ G_add_color_rule(-150, 0, 0, 255, -40, 0, 255, 255, colors);
+ G_add_color_rule(-40, 0, 255, 255, -20, 0, 255, 0, colors);
+ G_add_color_rule(-20, 0, 255, 0, 0, 255, 255, 0, colors);
+ G_add_color_rule(0, 255, 255, 0, 20, 0, 255, 0, colors);
+ G_add_color_rule(20, 0, 255, 0, 40, 0, 255, 255, colors);
+ G_add_color_rule(40, 0, 255, 255, 150, 0, 0, 255, colors);
+ G_add_color_rule(150, 0, 0, 255, 200, 0, 0, 0, colors);
+
+ if (max 200)
+ G_add_color_rule(200, 0, 0, 0, max, 0, 0, 0, colors);
+
+ G_write_colors(wat_name, this_mapset, colors);
+}
if (asp_flag) {
cseg_write_cellfile(asp, asp_name);
G_init_colors(colors);
--- r.watershed2.orig/ram/close_maps.c 2008-11-29 11:45:15.0 +0100
+++ r.watershed2/ram/close_maps.c 2008-12-02 11:11:59.0 +0100
@@ -9,6 +9,8 @@
struct Colors colors;
int value, r, c, fd;
CELL *buf = NULL;
+struct Range accRange;
+CELL min, max;
if (wat_flag || asp_flag || dis_flag || ls_flag || sl_flag || sg_flag)
buf = G_allocate_cell_buf();
@@ -30,6 +32,43 @@
}
if (G_close_cell(fd) 0)
[GRASS-dev] r.watershed2 with MFD
I took the request for MFD support in r.watershed by Helena and Dylan to heart and implemented it, but still need a few more days to clean up the code, then I want to submit it as r.watershed2.mfd to grass-addons. A first result for flow accumulation is here: http://markus.metz.giswork.googlepages.com/rw2_mfd_large.jpg This output has been calculated from elev_lid1972_1m in the North Carolina sample dataset. Please compare to the original result and flow accumulation of other modules as available here: http://www.grassbook.org/gallery/chapters/s050404f070_flow.jpg a) old r.watershed b) r.flow c) r.terraflow with MFD d) r.sim.water References: original MFD algorithm, also referenced to by r.terraflow Quinn, P., Beven, K., Chevallier, P., and Planchon, 0. 1991. The prediction of hillslope flow paths for distributed hydrological modelling using digital terrain models, Hydrol. Process., 5, 59-79. The algorithm of Quinn et al. (1991) was modified by Holmgren (1994) and is implemented here: Holmgren, P. 1994. Multiple flow direction algorithms for runoff modelling in grid based elevation models: an empirical evaluation. Hydrol. Process., 8, 327-334. I added a convergence factor after Holmgren (1994): a lower convergence factor results in more diverging flow, output will be similar to r.terraflow in MFD, a higher convergence factor will result in more converging flow, getting more similar to, but not identical to SFD. Additional memory requirements are not noticeable ( 2 KB) and independent of the region size, additional time required is a few seconds for elev_lid1972_1m, increasing by a few more seconds with higher convergence factor. The color coding suggested by Hamish with absolute logarithm works nice for most outputs, I have hardcoded it for flow accumulation output. The color coding of the example image above has been adjusted a bit to show more detail, but abs log of rainbow looks nice too! Markus Metz ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] r.watershed2 with MFD
Sorry, a mistake in the additional time requirements for MFD on elev_lid1972_1m: r.watershed2 with SFD: 0.7 seconds r.watershed2 with MFD: 1.0 seconds Markus Metz wrote: I took the request for MFD support in r.watershed by Helena and Dylan to heart and implemented it, but still need a few more days to clean up the code, then I want to submit it as r.watershed2.mfd to grass-addons. A first result for flow accumulation is here: http://markus.metz.giswork.googlepages.com/rw2_mfd_large.jpg This output has been calculated from elev_lid1972_1m in the North Carolina sample dataset. Please compare to the original result and flow accumulation of other modules as available here: http://www.grassbook.org/gallery/chapters/s050404f070_flow.jpg a) old r.watershed b) r.flow c) r.terraflow with MFD d) r.sim.water References: original MFD algorithm, also referenced to by r.terraflow Quinn, P., Beven, K., Chevallier, P., and Planchon, 0. 1991. The prediction of hillslope flow paths for distributed hydrological modelling using digital terrain models, Hydrol. Process., 5, 59-79. The algorithm of Quinn et al. (1991) was modified by Holmgren (1994) and is implemented here: Holmgren, P. 1994. Multiple flow direction algorithms for runoff modelling in grid based elevation models: an empirical evaluation. Hydrol. Process., 8, 327-334. I added a convergence factor after Holmgren (1994): a lower convergence factor results in more diverging flow, output will be similar to r.terraflow in MFD, a higher convergence factor will result in more converging flow, getting more similar to, but not identical to SFD. Additional memory requirements are not noticeable ( 2 KB) and independent of the region size, additional time required is a few seconds for elev_lid1972_1m, increasing by a few more seconds with higher convergence factor. The color coding suggested by Hamish with absolute logarithm works nice for most outputs, I have hardcoded it for flow accumulation output. The color coding of the example image above has been adjusted a bit to show more detail, but abs log of rainbow looks nice too! Markus Metz ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] r.watershed2 with MFD
I have SVN commit access for grass-addons, is this also valid for grass7? BTW my nightmare is to screw up the svn history that's why I hesitate to submit directly to grass7 (or grass64). Helena Mitasova wrote: Markus, at this point, as you suggest, the best would be to work on this in grass7 - do you have SVN commit access? Helena On Dec 5, 2008, at 3:02 PM, Markus Metz wrote: Dylan Beaudette wrote: On Fri, Dec 5, 2008 at 1:55 AM, Markus Metz [EMAIL PROTECTED] wrote: I took the request for MFD support in r.watershed by Helena and Dylan to heart and implemented it, but still need a few more days to clean up the code, then I want to submit it as r.watershed2.mfd to grass-addons. Is there any way to cleanly merge this with the existing r.watershed code in SVN so that we can test it? Also considering Michael's remarks, I suggets that I then not submit to grass-addons, but to grass 7. Not to grass-6.4.x, because this code is now more experimental and documentation still needs to be written. The MFD algorithm seems to be robust and produces the expected results with nice differences according to the convergence factor, but the other outputs are different and I don't know yet if this is ok and desired or not. The basins and half-basins are different, less so for coarser resolution, rather disastrous with MFD for the 1m LIDAR DEM in the North Carolina dataset. It seems that such a DEM should be processed with coarser resolution to obtain basins and half-basins that make sense, apparently both for SFD and MFD, but these a just first impressions. You may be using a threshold that is too small. Slope length (LS for USLE) and slope steepness (S for USLE) are about 99.9% identical between SFD and MFD with the DEM elevation in the North Carolina dataset. Streams are again different, i.e. MFD streams need to be thinned then they are very similar to SFD, but MFD streams make sense. Once I have added MFD to segmented mode too and updated the documentation, the module is ready to be submitted and scrutinised by others. This is no easy feat to modify r.watershed, I want to make sure the old behaviour is preserved when adding new functionality and I need some more time before submitting. I don't like to submit code with bugs... Markus Metz ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] error in georeferencing points on import with v.in.ascii
On Sun, Jun 9, 2013 at 2:50 AM, Hamish hamis...@yahoo.com wrote: Michael wrote: Here is the command: v.in.ascii --overwrite input=/Users/cmbarton/sites_out /sites_out.csv output=sites_test separator=, skip=1 x=15 y=16 cat=0 Here is the csv file: (got it) Once imported, take a look at the sites in the lower left. You can see the pairs of sites that should be the same location. seems that it's ok, v.out.ascii looks good. I think the GUI query tool is just picking up the nearby cat 15 at -30626,4116558, versus cat 46 and 47 at -31415|4116150. Works for me too, cats 46 and 47 are exactly in top of each other. The grid in the attached screenshot is a 1 cm grid. Markus M see attached screenshot. the two coords you posted are the two triangles. (d.mark addon) as for if the wx query tool will show you attribute data for both overlapping points (like QGIS does), I don't know. the wx digitizer has a threshold option for snapping, there doesn't seem to be a similar mouse-proximity adjustment for the query tool. maybe that doesn't need an adjustment, just a change from 10px to 5px (even better if it could deal with diff't screen DPIs) Hamish ps- background (etopo1) is a bit crude as r.in.onearth stopped working again/JPL changed their tiled WMS again? pps- er, the vector query tool defaults to edit mode? ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev attachment: points_grid.png___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] trouble compiling v.in.ogr + math.h (all branches)
On Mon, Jun 10, 2013 at 3:11 PM, Glynn Clements gl...@gclements.plus.com wrote: I'd suggest using: int exp; new_snap = frexp(xmax, exp); exp -= 52; new_snap = ldexp(new_snap, exp); frexp() and ldexp() are C89, don't introduce rounding errors, and handle zero correctly (they're also likely to be more efficient, but that's a trivial detail). Thanks for the suggestion, done in all branches. Markus M ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] bug in vector network analysis node costs
On Mon, Jun 10, 2013 at 10:29 PM, Štěpán Turek stepan.tu...@seznam.cz wrote: Hi all, Probably I found bug in nodes cost in vector network analysis (tested in G7, probably it affects all branches). Problem is in lib/vector/Vlib/net.c: int cost; ... (* dglInt32_t)(dglInt32_t) cost int cost is cast into dglInt32_t pointer. DGLib dglInt32_t is long. I am relatively new to C so I do not understand the way it is casted and I can miss something. I am reading it as cast address of cost int to dglInt32_t and then to pointer to dglInt32_t. It does not make sense to me but it compiles :-) The problem is when you have on your system different size of long and int, it is probably also the reason why it was not noticed so far (on 32 bit usually sizeof(int) == sizeof(long)). Because it seems that during this cast there are added bytes to be same size as long. However these bytes have random values and therefore results costs do not make sense. I think the reason why this has not been noticed yet is that not many users are working with node costs. 64 bit systems with sizeof(long) sizeof(int) are commonly used for quite a few years by now. The type name dglInt32_t is confusing because it is not necessarily a 32bit integer, it is an integer with at least 32 bits. If this change is made everything seems ok: dglInt32_t dgl_cost; ... dgl_cost = cost; dglNodeSet_Attr(gr, dglGetNode(gr, (dglInt32_t) cat), dgl_cost); Now there is: dglNodeSet_Attr(gr, dglGetNode(gr, (dglInt32_t) cat), (* dglInt32_t)(dglInt32_t) cost); I have attached also patch. Makes sense. I have applied the patch to all branches in r56680-2. Markus M ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] spline interpolation over time series
On Sun, Jun 16, 2013 at 8:50 PM, Sören Gebbert soerengebb...@googlemail.com wrote: Hi, Nikos is absolutely right, the ideal candidate for spline interpolation in time for raster maps is r.series.interp. There is no other module that performs temporal interpolation. r.hants does temporal interpolation, to a degree. Assuming you have e.g the raster maps tseries.1, tseries.3, tseries.5, tseries.7, tseries.9, you could create dummy tseries maps tseries.2, tseries.4, tseries.6, tseries.7 with r.mapcalc tseries.2 = null() etc, then r.hants in=tseries.1,tseries.2,tseries.3,tseries.4,tseries.5,tseries.6,tseries.7,tseries.8,tseries.9 with the appropriate filtering options. Markus M Best regards Soeren 2013/6/16 Nikos Alexandris n...@nikosalexandris.net Margherita Di Leo wrote: Hi all! I need to analyse several MODIS in time, and calculate the temporal spline interpolation (for each pixel), then pick the values (date and value) on the spline function corresponding to percentiles 25, 50, 75, maximum, minimum and average. I know that with r.series I can get max, min and average, but I can't currently apply a spline interpolation. I'd like to know if in the temporal modules there's something to do that or I should hack the r.series in order to add the spline interpolation. Maybe r.series.interp is a good/the proper candidate to receive such an addition? Nikos I found this http://docs.scipy.org/doc/scipy/reference/tutorial/interpolate.html#spline-i nterpolation that I can probably use for the scope. ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] Unexpected EVI range from i.vi
Feeding the test values and the evi(2) formula to r.mapcalc, the results are more or less in the expected range, still beyond [-1, 1], but not much. Obviously, there is a bug in i.vi. Furthermore, i.vi is slower than r.mapcalc. I recommend to replace the C module i.vi with a i.vi script that calls r.mapcalc. Markus M On Sun, Jun 16, 2013 at 8:13 PM, Nikos Alexandris n...@nikosalexandris.net wrote: Nikos Alexandris wrote: I get an unexpected range of evi values using i.vi (G7), i.e. [..] # range is... r.info -r evi min=-6912.82161611806 max=2264.42037461018 [..] Looking for these pixels, e.g. r.stats evi_ToAR -g | grep '\-5905' 784935 2695215 -5905.4417191748 786465 2694675 -5905.4417191748 766185 2685615 -5905.4417191748 and r.stats evi_ToAR -g | grep '6952\.' 783435 2690025 6952.8054373157 Then tracing the values that produced this out-of-range value, r.what coordinates=784935,2695215 map=B.Trimmed.ToAR.1,B.Trimmed.ToAR.3,B.Trimmed.ToAR.4,evi_DN,evi_ToAR 784935|2695215||0.230260364323039|0.110923862670943|0.0614305088322746| 0.35181236673774|-5905.44171917482 and r.what coordinates=783435,2690025 map=B.Trimmed.ToAR.1,B.Trimmed.ToAR.3,B.Trimmed.ToAR.4,evi_DN,evi_ToAR 783435|2690025||0.228143006540123|0.106632395012542|0.0712654621906476| 0.296610169491526|6952.80543731566 A bit more clear perhaps: # res=30 g.region -pagc rast=B.Trimmed.ToAR.1 res=30 n=2815500 s=2621340 w=658560 e=875880 nsres=30 ewres=30 rows=6472 cols=7244 cells=46883168 center_easting=767220.00 center_northing=2718420.00 # what is there? r.what coordinates=784935,2695215 map=B.Trimmed.ToAR.1,B.Trimmed.ToAR.3,B.Trimmed.ToAR.4,evi_DN,evi_ToAR 784935|2695215||0.230260364323039|0.110923862670943|0.0614305088322746| 0.35181236673774|-5905.44171917482 # while res=1... g.region -pagc rast=B.Trimmed.ToAR.1 res=1 n=2815500 s=2621340 w=658560 e=875880 nsres=1 ewres=1 rows=194160 cols=217320 cells=42194851200 center_easting=767220.00 center_northing=2718420.00 # ...gives r.what coordinates=784935,2695215 map=B.Trimmed.ToAR.1,B.Trimmed.ToAR.3,B.Trimmed.ToAR.4,evi_DN,evi_ToAR 784935|2695215||0.230260364323039|0.110923862670943|0.0614305088322746| 0.35181236673774|-5905.44171917482 Now, pointing to the suspect pixel: # both res=30res=1 give the same result, of course g.region -pagc e=784935 n=2695215 rows=1 cols=1 res=30 n=2695230 s=2621340 w=658560 e=784950 nsres=73890 ewres=126390 rows=1 cols=1 cells=1 center_easting=721755.00 center_northing=2658285.00 # what is there? r.what coordinates=784935,2695215 map=B.Trimmed.ToAR.1,B.Trimmed.ToAR.3,B.Trimmed.ToAR.4,evi_DN,evi_ToAR 784935|2695215||0.264138088849702|0.372703389833447| 0.438437054236573|-0.203045685279188|0.0970312073830427 [..] I guess it has to do with the extent/res definitions... Thanks, Nikos ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] spline interpolation over time series
On Sun, Jun 16, 2013 at 10:20 PM, Margherita Di Leo dileomargher...@gmail.com wrote: Hi, On Sun, Jun 16, 2013 at 9:19 PM, Markus Metz markus.metz.gisw...@gmail.com wrote: On Sun, Jun 16, 2013 at 8:50 PM, Sören Gebbert soerengebb...@googlemail.com wrote: Hi, Nikos is absolutely right, the ideal candidate for spline interpolation in time for raster maps is r.series.interp. There is no other module that performs temporal interpolation. r.hants does temporal interpolation, to a degree. Assuming you have e.g the raster maps tseries.1, tseries.3, tseries.5, tseries.7, tseries.9, you could create dummy tseries maps tseries.2, tseries.4, tseries.6, tseries.7 with r.mapcalc tseries.2 = null() etc, then r.hants in=tseries.1,tseries.2,tseries.3,tseries.4,tseries.5,tseries.6,tseries.7,tseries.8,tseries.9 with the appropriate filtering options. Thank you for all your inputs. Actually what I should do is not a proper interpolation but a regression That sounds like r.series (one time series) or r.regression.series (regression of two time series). Markus M -- Best regards, Margherita DI LEO Postdoctoral Researcher European Commission - DG JRC Institute for Environment and Sustainability (IES) Via Fermi, 2749 I-21027 Ispra (VA) - Italy - TP 261 Tel. +39 0332 78 3600 margherita.di-...@jrc.ec.europa.eu Disclaimer: The views expressed are purely those of the writer and may not in any circumstance be regarded as stating an official position of the European Commission. ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] Unexpected EVI range from i.vi
On Mon, Jun 17, 2013 at 10:58 PM, Nikos Alexandris n...@nikosalexandris.net wrote: Markus Metz wrote: Feeding the test values and the evi(2) formula to r.mapcalc, the results are more or less in the expected range, still beyond [-1, 1], but not much. [cut] Yes, but feeding the suspect values in r.mapcalc, still gives, correctly, large/out of range (regarding EVI's expected range) result(s). Just to confirm, the suspect values are blue - 0.230260364323039 red - 0.110923862670943 nir - 0.0614305088322746 evi - ( 2.5 * ( nir - red) ) / (nir + 6 * red - 7 * blue + 1.0) evi [1] -1.07453 and blue - 0.228143006540123 red - 0.106632395012542 nir - 0.0712654621906476 evi - ( 2.5 * ( nir - red) ) / (nir + 6 * red - 7 * blue + 1.0) evi [1] -0.7751909 ? Using r.mapcalc I get the same results like in R, not -5905.44171917482 or 6952.80543731566, respectively. Why are the way out of range values correct if R produces reasonable results? Markus M Anyhow, just to have a quick-check on r.what, should I upload the bands in question somewhere? Would anyone have the time to explain/check why r.what gives different results depending on the extent/resolution for the same coordinates? Which, might be expected, but why does g.region rows=1 cols=1 set the resolution to... see below: That explained Moritz. I would only add that you should align the region to the raster (not the raster's resolution, these are two different things), before using r.what, otherwise the raster lib will do some internal nn resamplng (if the region's resoltion is larger than the raster's resolution). Nikos Nikos Alexandris: [snip] Now, pointing to the suspect pixel: # both res=30res=1 give the same result, of course g.region -pagc e=784935 n=2695215 rows=1 cols=1 res=30 ---^--^-^^ n=2695230 s=2621340 w=658560 e=784950 --vv nsres=73890 ewres=126390 --^^ rows=1 cols=1 cells=1 center_easting=721755.00 center_northing=2658285.00 # what is there? r.what coordinates=784935,2695215 map=B.Trimmed.ToAR.1,B.Trimmed.ToAR.3,B.Trimmed.ToAR.4,evi_DN,evi_ToAR 784935|2695215||0.264138088849702|0.372703389833447| 0.438437054236573|-0.203045685279188|0.0970312073830427 [..] I guess it has to do with the extent/res definitions... ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] compilation of grass on AIX 7.1
On Wed, Jun 19, 2013 at 6:10 AM, Hamish hamis...@yahoo.com wrote: Markus N wrote: - diglib test updated in lib/vector/diglib/ -- still failing there, we try to understand That has been fixed in 56699, it was related to LFS. hmm, this is quite similar, I wonder if it is related: grass: FTBFS on ppc64 and s390x http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=672719#17 Errors in: /build/buildd-grass_6.4.1-2.1-s390x-GTALEw/grass-6.4.1/lib/vector/diglib ... cd OBJ.s390x-ibm-linux-gnu; \ LD_LIBRARY_PATH=:/build/buildd-grass_6.4.1-2.1-s390x-GTALEw/grass-6.4.1/dist.s390x-ibm-linux-gnu/lib ./test; diff ./test.tmp ../ test.ok Killed make[6]: *** [OBJ.s390x-ibm-linux-gnu/test] Error 2 That seems to me a problem on big endian systems, not related to LFS, fixed after 6.4.1. I would recommend to upgrade GRASS to either the latest 6.4.3 snapshot or to 7.0. Markus M ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] Unexpected EVI range from i.vi
I think I figured it out: The EVI formula in i.vi is for MODIS. The generic formula is G * ( nir - red) / (nir + C1 * red - C2 * blue + L) where G is a gain factor, C1, C2 are coefficients to correct for aerosol influences in the red band using the blue band and L is the canopy background adjustment that addresses non-linear, differential NIR and red radiant transfer through a canopy. The formula is scale-dependent, i.e. dependent on the range of the input data. Scaling the input test values to a [0, 1] range (the range of the MODIS input bands), I get EVI values in the range [-1, 1]. Furthermore, different coefficients might be needed for different sensors. Some more points: The reference in i.vi for EVI is wrong, it must be A.Huete, K.Didan, T.Miura, E.P.Rodriguez, X.Gao, L.G. Ferreira. 2002. Overview of the radiometric and biophysical performance of the MODIS vegetation indices. Remote Sensing of Environment 83 195-213. The formula we used in R is wrong: evi - ( 2.5 * ( nir - red) ) / (nir + 6 * red - 7 * blue + 1.0) it must be evi - ( 2.5 * ( nir - red) ) / (nir + 6 * red - 7.5 * blue + 1.0) then I get again 6952.805 g.region rast= res= (-a) does not make sense, unless you want to resample a raster map. Markus M ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] Unexpected EVI range from i.vi
On Thu, Jun 20, 2013 at 6:24 PM, Nikos Alexandris n...@nikosalexandris.net wrote: Markus Metz wrote: I think I figured it out: The EVI formula in i.vi is for MODIS. That's precise, EVI is MODIS specific. We should clearly describe this in the manual (I will try to alter the respective text). From the literature, I gor the impression that EVI can be calculated from other sensors as well, as long as you get the coefficients right. The generic formula is G * ( nir - red) / (nir + C1 * red - C2 * blue + L) where G is a gain factor, C1, C2 are coefficients to correct for aerosol influences in the red band using the blue band and L is the canopy background adjustment that addresses non-linear, differential NIR and red radiant transfer through a canopy. Assuming that the input to i.vi should be properly preprocessed bands with a theoretically maximum range of [0, 1], you could set L to 0.0001 and would get reasonable EVI values, sensor-independent. Markus M ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] g.proj not found
On Fri, Jun 21, 2013 at 9:58 AM, Luca Delucchi lucadel...@gmail.com wrote:
Hi devs,
just update grass7 and the gui doesn't start, the problem seems
related to missing module g.proj, but I think probably is related to
find_program.
find_program('r.proj','help') and also find_program('r.proj') return
false instead find_program('ls') return true
the error is
GRASS module 'g.proj' not found. Unable to start map display window.
Fixed in r56865.
Markus M
___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] Unexpected EVI range from i.vi
On Fri, Jun 21, 2013 at 10:04 PM, Nikos Alexandris n...@nikosalexandris.net wrote: Markus Metz wrote: I think I figured it out: The EVI formula in i.vi is for MODIS. Nikos A: That's precise, EVI is MODIS specific. We should clearly describe this in the manual (I will try to alter the respective text). MM: From the literature, I gor the impression that EVI can be calculated from other sensors as well, as long as you get the coefficients right. Yes, but this is not an easy job, is it? This is (also) why, I think, they (MODIS science team) developed the EVI2, which is cross-sensor. I am pretty sure that the EVI2 formula in i.vi is not cross-sensor, but also tailored to unscaled MODIS input bands: EVI2 = G * ( nir - red) / (nir + C1 * red + L) again, L needs to be adjusted to the actual input scale. AFAIU, both EVI and EVI2 can be applied to different sensors, given that the required bands are available and that the coefficients are adjusted if need be. The generic formula is G * ( nir - red) / (nir + C1 * red - C2 * blue + L) where G is a gain factor, C1, C2 are coefficients to correct for aerosol influences in the red band using the blue band and L is the canopy background adjustment that addresses non-linear, differential NIR and red radiant transfer through a canopy. Assuming that the input to i.vi should be properly preprocessed bands with a theoretically maximum range of [0, 1], you could set L to 0.0001 and would get reasonable EVI values, sensor-independent. This reminds, if I am not wrong (didn't check) the scale factor for MOD09 surface reflectance products. Makes sense. I suggested L = 0.0001 exactly because this is the MODIS scale factor. BTW, the satellite data you mentioned are ETM, not MODIS, thus applying the EVI formula developed for MODIS to ETM bands is a bit adventurous. In any case, EVI was developed for tropical rain forests because NDVI can saturate there. The Landsat scene you mentioned has only ocean and desert, no forest of any kind. NDVI should be just fine in this area. I would suggest to test the EVI(2) formulas in i.vi with a MODIS NDVI product which also includes the required input bands. All bands in the MODIS NDVI product would need to be scaled according to the documentation prior to feeding them to i.vi, or r.mapcalc with adjusted formulas. Markus M ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] Unexpected EVI range from i.vi
Markus Metz wrote: I am pretty sure that the EVI2 formula in i.vi is not cross-sensor, but also tailored to unscaled MODIS input bands: EVI2 = G * ( nir - red) / (nir + C1 * red + L) Hmm, no, it seems that the coefficients for both EVI and EVI2 are meant for scaled MODIS input bands (top of atmosphere reflectance). The coefficients for EVI are a mystery. The MODIS algorithm theoretical basis document refers to Huete and Liu (1996), which does not exist in the references. Huete et al (2002) specifies L = 1, C1 = 6, C2 = 7.5, and G = 2.5, referring to Huete, Justice, Liu (1994) and Huete, Liu, Batchily, van Leeuwen (1997), but none of these references explain how these numbers were determined. Interestingly, the MODIS algorithm theoretical basis document sets G = 2, not 2.5, without any explanation. In order to get an EVI in the range [-1, 1], using EVI = G * (nir - red) / (nir + C1 * red - C2 * blue + L) (nir + C1 * red - C2 * blue + L) must be = G * (nir - red) which could easily be added to i.vi. In any case, L needs to be adjusted to the actual input scale. In summary, rather use NDVI. Markus M ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] compilation of grass on AIX 7.1
On Wed, Jun 26, 2013 at 7:40 PM, Markus Neteler nete...@osgeo.org wrote: On Wed, Jun 26, 2013 at 1:13 AM, Glynn Clements gl...@gclements.plus.com wrote: Markus Neteler wrote: -bash-3.2$ cd /gpfs/home/neteler/software/grass-7.0.svn/raster/r.terraflow ld: 0711-317 ERROR: Undefined symbol: .std::basic_filebufchar, std::char_traitschar ::open64(char const*, std::_Ios_Openmode) This appears to be a problem with g++ on AIX, caused by AIX having #define open open64 in one of its headers: http://gcc.gnu.org/ml/gcc-patches/2012-09/msg01957.html It might be possible to work around this with e.g.: #include fcntl.h #undef open Markus Metz figured it out, fixed locally (see attachment). Submit or not? The fix is based on the hack proposed in http://gcc.gnu.org/ml/gcc-patches/2012-09/msg01957.html which is IMHO a hack, not a fix. Anyway, the hack has been applied to later versions of gcc. Therefore I would suggest to not submit the local hack for r.terraflow, but suggest to add the patch to http://grasswiki.osgeo.org/wiki/Compile_and_Install#AIX Anybody who wants to compile C++ modules using open() on AIX with LFS and a buggy gcc can then apply the patch. The patch seems to be too specialized to warrant submission to trunk even though no fixed gcc seems to be available for AIX, yet. Markus M PS: AIX is apparently deviating from UNIX standards so far that it can hardly be called a UNIX system any more... GRASS' LFS support on AIX7 also fixed also in http://trac.osgeo.org/grass/changeset/56936 Now trying to get shared libraries enabled (almost there). Markus ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] compilation of grass on AIX 7.1
On Fri, Jun 28, 2013 at 7:13 AM, Markus Neteler nete...@osgeo.org wrote: On Wed, Jun 26, 2013 at 7:40 PM, Markus Neteler nete...@osgeo.org wrote: ... Now trying to get shared libraries enabled (almost there). Also enabled in SVN (thanks to Markus Metz). Interestingly, when *starting* GRASS 7, I get on that AIX machine: Cleaning up temporary files... Starting GRASS GIS... Could not load program g.gisenv: Could not load module /gpfs/home/neteler/software/grass-7.0.svn/dist.powerpc-ibm-aix7.1.0.0/lib/libgrass_gis.7.0.svn.so. Dependent module /usr/lib/libiconv.a(libiconv.so.2) could not be loaded. Member libiconv.so.2 is not found in archive Could not load module g.gisenv. Dependent module /gpfs/home/neteler/software/grass-7.0.svn/dist.powerpc-ibm-aix7.1.0.0/lib/libgrass_gis.7.0.svn.so could not be loaded. Could not load module . Could not load program g.gisenv: Could not load module /gpfs/home/neteler/software/grass-7.0.svn/dist.powerpc-ibm-aix7.1.0.0/lib/libgrass_gis.7.0.svn.so. Dependent module /usr/lib/libiconv.a(libiconv.so.2) could not be loaded. Member libiconv.so.2 is not found in archive Could not load module g.gisenv. Dependent module /gpfs/home/neteler/software/grass-7.0.svn/dist.powerpc-ibm-aix7.1.0.0/lib/libgrass_gis.7.0.svn.so could not be loaded. Could not load module . Indeed, the libconv stuff is in /opt/freeware/ but searched in the common /usr/ directories. Existing: /usr/include/iconv.h /* @(#)09 1.20 src/bos/usr/include/iconv.h, libiconv, bos610 8/28/03 11:09:44 */ /* IBM_PROLOG_END_TAG */ ... but no corresponding libiconv. How to tell GRASS 7 in configure to pick up the /opt/freeware/ include file and library? In general, you can use some influential environment variables: CFLAGS C compiler flags LDFLAGS linker flags, e.g. -Llib dir if you have libraries in a nonstandard directory lib dir LIBSlibraries to pass to the linker, e.g. -llibrary CPPFLAGS(Objective) C/C++ preprocessor flags, e.g. -Iinclude dir if you have headers in a nonstandard directory include dir If some components are in non-standard /opt/freeware/, you could try to set LDFLAGS=-bsvr4 -R/opt/freeware/lib CPPFLAGS=-I/opt/freeware/include Markus M ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] compilation of grass on AIX 7.1
On Fri, Jun 28, 2013 at 9:09 PM, Markus Neteler nete...@osgeo.org wrote: On Fri, Jun 28, 2013 at 8:29 PM, Markus Metz markus.metz.gisw...@gmail.com wrote: On Fri, Jun 28, 2013 at 7:13 AM, Markus Neteler nete...@osgeo.org wrote: ... How to tell GRASS 7 in configure to pick up the /opt/freeware/ include file and library? In general, you can use some influential environment variables: CFLAGS C compiler flags LDFLAGS linker flags, e.g. -Llib dir if you have libraries in a nonstandard directory lib dir LIBSlibraries to pass to the linker, e.g. -llibrary CPPFLAGS(Objective) C/C++ preprocessor flags, e.g. -Iinclude dir if you have headers in a nonstandard directory include dir If some components are in non-standard /opt/freeware/, you could try to set LDFLAGS=-bsvr4 -R/opt/freeware/lib CPPFLAGS=-I/opt/freeware/include Not on AIX it seems...: checking whether the C compiler (gcc -bsvr4 -R/opt/freeware/lib) works... no configure: error: installation or configuration problem: C compiler cannot create executables. cat config.log ... configure:754: checking host system type configure:777: checking for gcc configure:881: checking whether the C compiler (gcc -bsvr4 -R/opt/freeware/lib) works configure:897: gcc -o conftest -I/opt/freeware/include -bsvr4 -R/opt/freeware/lib conftest.c 15 gcc: error: unrecognized option '-bsvr4' gcc: error: unrecognized option '-R' ... Oops, LDFLAGS should probably be LDFLAGS=-Wl,-bsvr4,-R,/opt/freeware/lib -L/opt/freeware/lib For comparison, on Solaris, these extra packages including gdal are in /opt/csw and I need to set LDFLAGS=-Wl,-R/opt/csw/lib -L/opt/csw/lib -Wl,-R/opt/csw/gxx/lib -L/opt/csw/gxx/lib in order to 1) get GRASS compiled, 2) get GRASS running, because anything in /opt/csw should only be used by applications compiled against it. By default, the system does not know about applications, libraries or headers in /opt/csw, and I guess that this principle (hide extra packages from the system) is common for both UNIX systems, AIX and Solaris. Sigh :-) Alternatively, try to disable nls, i.e. do not --enable-nls during configuration. Markus M ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] G_distance( ) problem in r.slope.aspect?
Try r.in.gdal input=as_dem_30s.bil output=dem_asia_hydrosheds_30s # check extents and range r.info dem_asia_hydrosheds_30s # set the region g.region -p rast=dem_asia_hydrosheds_30s r.colors map=dem_asia_hydrosheds_30s rules=srtm r.slope.aspect elevation=dem_asia_hydrosheds_30s slope=dem_asia_hydrosheds_30s_slope format=percent r.colors map=dem_asia_hydrosheds_30s_slope color=slope # check result r.info dem_asia_hydrosheds_30s_slope Markus M On Sat, Jun 29, 2013 at 7:14 AM, Yann Chemin yche...@gmail.com wrote: Hi, data: --- http://hydrosheds.cr.usgs.gov/datadownload.php?reqdata=30demb script: r.in.bin --overwrite input=/home/yann/RSDATA/hydrosheds/as_dem_30s.bil output=dem_asia_hydrosheds_30s bytes=2 north=59.715 south=-10.0 east=179.5 west=55.0 rows=8400 cols=15000 anull=- r.null map=dem_asia_hydrosheds_30s setnull=55537 r.colors map=dem_asia_hydrosheds_30s rules=srtm r.slope.aspect --overwrite elevation=dem_asia_hydrosheds_30s@PERMANENT slope=dem_asia_hydrosheds_30s_slope format=percent r.colors map=dem_asia_hydrosheds_30s_slope@PERMANENT color=slope complaint: -- WARNING: Unable to read fp range file for dem_asia_hydrosheds_30s_slope@PERMANENT Color table for raster map dem_asia_hydrosheds_30s_slope@PERMANENT set to 'slope' r.info on map: -- . ... Range of data:min = -nan max = -nan ... . -- ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] i.segment error: Ri is 0
On Mon, Jul 1, 2013 at 3:56 PM, Moritz Lennert mlenn...@club.worldonline.be wrote: Hi Eric and Markus, Trying to use i.segment in grass7 checked out and compiled a few days ago (rev 56918), I came upon the following error and the resulting segments file was not created. I can file this as a bug report, but wanted your feedback first to see if I'm misusing i.segment somehow. I haven't been able to find the error in the source code. command line: time i.segment group=xs out=seg_xs minsize=2 memory=3072 threshold=0.2 --o error: Segmentation converged after 16 iterations. Merging segments smaller than 2 cells ERREUR :Ri is 0 This should not happen. The ID of a segment is always positive or negative or NULL (Rast_is_c_null_value()) This is on a mosaic of Worldview 2 images with region specs as follows: g.region -p projection: 1 (UTM) zone: 33 datum: wgs84 ellipsoid: wgs84 north: 4876400 south: 4849792 west: 610056 east: 634648 nsres: 2 ewres: 2 rows: 13304 cols: 12296 cells: 163585984 The mosaic is only a narrow band within that region, so that actually there are only 34,755,878 non-null cells. Any hints ? Not really. I created a sample dataset with a MASK leaving only a narrow diagonal strip and everything went fine. Did you get any other warnings while running i.segment? I assume you are using 64 bit Linux with more than 3072 GB RAM and lots of free disk space on the partition with your GRASS data. Can you provide data to replicate or commands using one of the sample datasets to replicate this error? Markus M ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] i.segment error: Ri is 0
On Tue, Jul 2, 2013 at 9:58 AM, Moritz Lennert mlenn...@club.worldonline.be wrote: On 01/07/13 21:04, Markus Metz wrote: On Mon, Jul 1, 2013 at 3:56 PM, Moritz Lennert mlenn...@club.worldonline.be wrote: Hi Eric and Markus, Trying to use i.segment in grass7 checked out and compiled a few days ago (rev 56918), I came upon the following error and the resulting segments file was not created. I can file this as a bug report, but wanted your feedback first to see if I'm misusing i.segment somehow. I haven't been able to find the error in the source code. command line: time i.segment group=xs out=seg_xs minsize=2 memory=3072 threshold=0.2 --o error: Segmentation converged after 16 iterations. Merging segments smaller than 2 cells ERREUR :Ri is 0 This should not happen. The ID of a segment is always positive or negative or NULL (Rast_is_c_null_value()) This is on a mosaic of Worldview 2 images with region specs as follows: g.region -p projection: 1 (UTM) zone: 33 datum: wgs84 ellipsoid: wgs84 north: 4876400 south: 4849792 west: 610056 east: 634648 nsres: 2 ewres: 2 rows: 13304 cols: 12296 cells: 163585984 The mosaic is only a narrow band within that region, so that actually there are only 34,755,878 non-null cells. Any hints ? Not really. I created a sample dataset with a MASK leaving only a narrow diagonal strip and everything went fine. I did not use a mask, the non-relevant cells were just nulls. Would it be better to create a mask ? No, i.segment tests with Rast_is_d_null_value() which is true for both NULL cells and MASK'ed cells. Can you provide data to replicate or commands using one of the sample datasets to replicate this error? I'll run the segmentation again without having something else running. If I can reproduce the error, I'll make the data available to you so you can test. Other processes should not be able to interfere, as long as enough system resources are available which seems to be the case. I suspect a problem with nodata handling, but the code looks ok to me. If need be, you can send me (a link to) the test data off-list. Markus M ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] Status of r.cva?
On Tue, Jul 2, 2013 at 1:15 PM, Nikos Alexandris n...@nikosalexandris.net wrote: a friend needs to use r.cva [0,1] (and r.viewshed [2]). [2] Module/manual exists only for G7 http://grass.osgeo.org/grass70/manuals/r.viewshed.html, not for G64! r.viewshed is available as add-on for G6: http://grasswiki.osgeo.org/wiki/AddOns/GRASS_6#r.viewshed Markus M ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
[GRASS-dev] wingrass7 nightly builds broken
For the last two days, the nightly builds for trunk are broken. Configure fails [0] with [...] checking whether to use regex... yes checking for location of regex includes... checking for regex.h... no configure: error: *** Unable to locate regex includes. I guess some osgeo4w update moved or removed regex? Markus M [0] http://wingrass.fsv.cvut.cz/grass70/logs/log-r56987-632/package.log ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] wingrass7 nightly builds broken
2013/7/3 Jürgen E. j...@norbit.de: Hi Markus, On Wed, 03. Jul 2013 at 20:17:46 +0200, Markus Metz wrote: I guess some osgeo4w update moved or removed regex? AFAICT no. apps/msys/include/regex.h is in still in msys-1.0.11-13 (and that's from Apr 13th). Then it seems to be a problem on the server creating the nightly builds. Actually it affects all branches. Markus M ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] A few additional questions about i.segment
On Thu, Jul 4, 2013 at 11:13 AM, Moritz Lennert mlenn...@club.worldonline.be wrote: As you might have gathered from my other messages on the list, we are currently running tests of i.segment in our department. A few questions came up (with my attempts at answers). *** I had a look at the results, they are interesting. Maybe you could also try a combination of panchromatic, red and near-infrared bands and leave the other bands out. If it is possible to give different weights to the bands, then maybe 2 for the pan, 1 for red and 1 for NIR. I think that is possible, but I have to check how. I imagine you would work at the resolution of the pan for this... You could first resample the pan band to the resolution of the other bands, then r.mapcalc - multiply the pan band with 2 and use i.segment -w. Regarding the size of the segments, we generally combine different segmentations, fine and coarse, depending on the type of objects that we have to classify. We also combine different segmentations because then we can use the classification of a fine segmentation as attribute for classifying a coarse segmentation (e.g. counting trees or houses in a large segment). Here we have very small segments and also very large ones together in the results, it is a bit different. So to be able to do what you do now, you would need homogenous segment sizes, i.e. not only a minsize (which I set fairly low in this first trial: 2 pixels), but also the equivalent of a maxsize. Then, if you set the two fairly close to each other (i.e. minsize=900 and maxsize=1100 to get segments of +/- 1000 pixels). This is currently controlled with the threshold value. I am not sure if a max size makes sense. There might be homogenous objects with identical pixel values (difference = 0) that are larger than a given max size. Splitting such objects to respect a max size would be arbitrary. I would rather try to find a good combination of threshold and minsize, and do hierachical segmentation using the output of the previous run as input for the next run with increasing threshold. Technically, a maxsize option is possible for i.segment. Markus M I'll see how that can be achieved with the currently implemented module. *** Any hints on possible answers to these issues ? Moritz ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] v.in.ogr fails to import from PostGIS with schema
On Fri, Jul 5, 2013 at 10:02 AM, Blumentrath, Stefan stefan.blumentr...@nina.no wrote: Dear all I am reposting this message, because the problem, that I cannot import lines from PostGIS is still present in GRASS 7 svn-revision 57009 (and in 6.4.3svn (older revision) too.) Do you think this is only due to my PostGIS og GDAL/OGR installation, or shall I file a ticket? Please try trunk r57016. If this works for you, the changes should be backported. Another (much less important) thing here is, that the status message says: Counting polygons for 6125184 features (OGR layer n50_2013_utm33n.n50_2013_arealdekke_lines)... although the layer only contains (6125184) lines... The module only knows if a layer only contains lines after it went through all features. In this case the polygon count will be zero. Unless you use v.in.ogr type=boundary in which case lines will be converted to boundaries. Markus M Cheers Stefan -Original Message- From: grass-dev-boun...@lists.osgeo.org [mailto:grass-dev-boun...@lists.osgeo.org] On Behalf Of Blumentrath, Stefan Sent: 31. mai 2013 08:27 To: Markus Neteler; grass-dev@lists.osgeo.org Subject: Re: [GRASS-dev] v.in.ogr fails to import from PostGIS with schema Hi Markus Sorry, good to know. It was HTML indeed. So here is the problem-description in plain ASCII: When I try to import a vector map containing (lots of) lines from a certain schema in a PostGIS-database, i get the following error messages: v.in.ogr dsn=PG:host=myserver dbname=gisdata layer=n50_2013_utm33n.n50_2013_arealdekke_lines output=Test (...) DBMI-SQLite driver error: Error in sqlite3_prepare(): unrecognized token: 3insert DBMI-SQLite driver error: Error in sqlite3_prepare(): unrecognized token: 3insert ERROR: Cannot insert new row: insert into Test values ( 3insert into Test values ( 3, NULL, NULL, NULL, NULL, NULL, NULL, 'FiktivDelelinje', '1000-01-01 0:00:00', NULL, 101 ) While other formats with the same data work just fine: ogr2ogr -f SQLite /home/stefan/tmp/N50/n50_2013_arealdekke_lin.sqlite PG:host=myserver dbname=gisdata n50_2013_utm33n.n50_2013_arealdekke_lines v.in.ogr dsn=/home/stefan/tmp/N50/n50_2013_arealdekke_lin.sqlite output=N50_2013_arealdekke_linThis gave no errors... Forgot to mention in my earlier post, that also importing polygons from the same PostGIS database (and schema) as the lines works fine: v.in.ogr dsn=PG:host=myserver dbname=gisdata layer=n50_2013_utm33n.n50_2013_arealdekke_polygons output=Test_pol Gave no error messages either... So, maybe it is something with the geometry type? Cheers Stefan -Original Message- From: neteler.os...@gmail.com [mailto:neteler.os...@gmail.com] On Behalf Of Markus Neteler Sent: 31. mai 2013 00:22 To: Blumentrath, Stefan Subject: Re: [GRASS-dev] v.in.ogr fails to import from PostGIS with schema Hi Stefan, you message (HTML?) did not contain the relevant error... see below. It is better to send in pure ASCII... Best Markus On Thu, May 30, 2013 at 9:51 PM, SBL stefan.blumentr...@nina.no wrote: Hi When I try to to import a vector map containing (lots of) lines from a certain schema in a PostGIS-database, i get the following error messages: While other formats with the same data work just fine: This gave no errors... I was using GRASS 7 (compiled about a month ago) and PostGIS 2.0.1 / PostgreSQL 9.1 on Ubuntu 12.04 LTS Server. Looks like something got mixed up in the insert statement. Is that a known issue or shall I file a ticket? Cheers Stefan -- View this message in context: http://osgeo-org.1560.x6.nabble.com/v-in-ogr-fails-to-import-from-Post GIS-with-schema-tp5057034.html Sent from the Grass - Dev mailing list archive at Nabble.com. ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] v.in.ogr fails to import from PostGIS with schema
On Fri, Jul 5, 2013 at 11:25 AM, Blumentrath, Stefan stefan.blumentr...@nina.no wrote: Thanks Markus! It works now. OK, backported to 6.5 and 6.4 in r57017/8. Regarding the status message: I just found it a bit confusing, not more than that... Changed in trunk to Check if OGR layer n50_2013_utm33n.n50_2013_arealdekke_lines contains polygons... Markus M Stefan -Original Message- From: Markus Metz [mailto:markus.metz.gisw...@gmail.com] Sent: 5. juli 2013 10:51 To: Blumentrath, Stefan Cc: grass-dev@lists.osgeo.org Subject: Re: [GRASS-dev] v.in.ogr fails to import from PostGIS with schema On Fri, Jul 5, 2013 at 10:02 AM, Blumentrath, Stefan stefan.blumentr...@nina.no wrote: Dear all I am reposting this message, because the problem, that I cannot import lines from PostGIS is still present in GRASS 7 svn-revision 57009 (and in 6.4.3svn (older revision) too.) Do you think this is only due to my PostGIS og GDAL/OGR installation, or shall I file a ticket? Please try trunk r57016. If this works for you, the changes should be backported. Another (much less important) thing here is, that the status message says: Counting polygons for 6125184 features (OGR layer n50_2013_utm33n.n50_2013_arealdekke_lines)... although the layer only contains (6125184) lines... The module only knows if a layer only contains lines after it went through all features. In this case the polygon count will be zero. Unless you use v.in.ogr type=boundary in which case lines will be converted to boundaries. Markus M Cheers Stefan -Original Message- From: grass-dev-boun...@lists.osgeo.org [mailto:grass-dev-boun...@lists.osgeo.org] On Behalf Of Blumentrath, Stefan Sent: 31. mai 2013 08:27 To: Markus Neteler; grass-dev@lists.osgeo.org Subject: Re: [GRASS-dev] v.in.ogr fails to import from PostGIS with schema Hi Markus Sorry, good to know. It was HTML indeed. So here is the problem-description in plain ASCII: When I try to import a vector map containing (lots of) lines from a certain schema in a PostGIS-database, i get the following error messages: v.in.ogr dsn=PG:host=myserver dbname=gisdata layer=n50_2013_utm33n.n50_2013_arealdekke_lines output=Test (...) DBMI-SQLite driver error: Error in sqlite3_prepare(): unrecognized token: 3insert DBMI-SQLite driver error: Error in sqlite3_prepare(): unrecognized token: 3insert ERROR: Cannot insert new row: insert into Test values ( 3insert into Test values ( 3, NULL, NULL, NULL, NULL, NULL, NULL, 'FiktivDelelinje', '1000-01-01 0:00:00', NULL, 101 ) While other formats with the same data work just fine: ogr2ogr -f SQLite /home/stefan/tmp/N50/n50_2013_arealdekke_lin.sqlite PG:host=myserver dbname=gisdata n50_2013_utm33n.n50_2013_arealdekke_lines v.in.ogr dsn=/home/stefan/tmp/N50/n50_2013_arealdekke_lin.sqlite output=N50_2013_arealdekke_linThis gave no errors... Forgot to mention in my earlier post, that also importing polygons from the same PostGIS database (and schema) as the lines works fine: v.in.ogr dsn=PG:host=myserver dbname=gisdata layer=n50_2013_utm33n.n50_2013_arealdekke_polygons output=Test_pol Gave no error messages either... So, maybe it is something with the geometry type? Cheers Stefan -Original Message- From: neteler.os...@gmail.com [mailto:neteler.os...@gmail.com] On Behalf Of Markus Neteler Sent: 31. mai 2013 00:22 To: Blumentrath, Stefan Subject: Re: [GRASS-dev] v.in.ogr fails to import from PostGIS with schema Hi Stefan, you message (HTML?) did not contain the relevant error... see below. It is better to send in pure ASCII... Best Markus On Thu, May 30, 2013 at 9:51 PM, SBL stefan.blumentr...@nina.no wrote: Hi When I try to to import a vector map containing (lots of) lines from a certain schema in a PostGIS-database, i get the following error messages: While other formats with the same data work just fine: This gave no errors... I was using GRASS 7 (compiled about a month ago) and PostGIS 2.0.1 / PostgreSQL 9.1 on Ubuntu 12.04 LTS Server. Looks like something got mixed up in the insert statement. Is that a known issue or shall I file a ticket? Cheers Stefan -- View this message in context: http://osgeo-org.1560.x6.nabble.com/v-in-ogr-fails-to-import-from-Pos t GIS-with-schema-tp5057034.html Sent from the Grass - Dev mailing list archive at Nabble.com. ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev ___ grass-dev mailing list grass-dev@lists.osgeo.org http
Re: [GRASS-dev] [SoC] Weekly report #3 - GRASS Interactive Scatter Plot Tool
On Sat, Jul 6, 2013 at 4:19 PM, Štěpán Turek stepan.tu...@seznam.cz wrote: Hi Hamish and Michael, your questions are connected I will try to explain both. The scatter plot tool backend is run in same process as wxGUI. If you select some area in scatter plot, then data from numpy arrays are passed to the backend (through ctypes), which describes selected areas in scatters plots. You probably don't need ctypes for that, have a look at lib/python/script/array.py. If you want to use ctypes, keep in mind that GRASS is designed for modular use. Tools using library functions must run as a separate process, i.e. the wxGUI should run just fine also in the absence of ctypes. And the backend computes corresponding areas which will be highlighted in all opened scatter plots. To avoid frozing of the wxGUI, the computation is done in separate python thread. Currently r.out.bin is used to bypass inability to set region in raster library for different rasters. When user choose raster group for analysis in the tool, it is exported into binary files in current region and the backend works directly with the binary files. This is not very nice. If the backend would be written as module, then I would need to create temporary files to pass the selected areas, which does not seem very elegant to me. When processing raster data, temporary files are created pretty much all the time (by the raster lib). Regarding the scatterplot tool which works with raster data which can be very large, an advantage of temporary files is that out-of-memory errors are more easily avoided. In simplified way the idea is that the library should give you option whether you want to open and then work with raster file according to the statically set variables (in structure R__) or open it with dynamically defined variables. With regard to the current region, it seems to me that this would violate the concept of the current region. Rather run the scatterplot tool as a separate process, then you can modify the region without changing the regular current region. When using raster lib functions directly, maybe Rast_set_input_window() can help you? It could be also useful for Pietro's pyGRASS API. And it would be first small step to achieve compilation of raster modules as library instead of different programs. I am working on more concrete proposal of the changes in raster library during this weekend. Raster modules, as all GRASS modules, should (IMHO must definitively) stay modules because GRASS is designed for modular use. Modules can easily call other modules, effectively using them instead of some, just like some library function. Again, the advantage is that modules are run as a separate process. my 2c Markus M I am new to this problematic, so all these suggestions can be totally wrong.. Best Stepan -- Původní zpráva -- Od: Michael Barton michael.bar...@asu.edu Datum: 6. 7. 2013 Předmět: Re: [SoC] Weekly report #3 - GRASS Interactive Scatter Plot Tool So I take it you are not using r.stats to generate the data to plot, but are getting data directly accessing rasters via GRASS libraries? Michael __ Hi, I am interested to hear more about the library limitations.. you mean the WIND region, right? I'd be surprised if WIND_OVERRIDE or GRASS_REGION enviro variables (and the python wrappers for them) didn't provide full control for whatever someone might like to do. I would guess if the scatter tool was spawned as an independent process, it could maintain its own environment so it's own region override enviro variables. or simply unset the wxGUI display overrides and just use the mapset's real computational region read directly from the WIND file? regards, Hamish ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] Compiling grass7_trunk in Funtoo
On Sun, Jul 7, 2013 at 10:25 AM, Nikos Alexandris n...@nikosalexandris.net wrote: Hi devs! Hope that some advanced Gentoo user is tuned-in. I am trying to compile grass7_trunk in Funtoo. I managed to get a clean configuration of almost everything required (except for LAPACK, BLAS and FFMPEG, all of which, if I am no miss-taking, are not required). You can check for optional functionality with ./configure --help. AFAIK, BLAS and LAPACK are not used. The compilation ends-up like: --%--- Started compilation: Sun Jul 7 10:24:41 EEST 2013 -- Errors in: /osgeo/src/grass7_trunk/lib/python/ctypes /osgeo/src/grass7_trunk/db/drivers/ogr /osgeo/src/grass7_trunk/db/drivers/dbf /osgeo/src/grass7_trunk/db/drivers/postgres /osgeo/src/grass7_trunk/db/drivers/sqlite /osgeo/src/grass7_trunk/db/drivers/odbc /osgeo/src/grass7_trunk/db/drivers/mysql /osgeo/src/grass7_trunk/gui/wxpython/vdigit /osgeo/src/grass7_trunk/gui/wxpython/mapswipe /osgeo/src/grass7_trunk/gui/wxpython/rlisetup /osgeo/src/grass7_trunk/gui/wxpython/animation /osgeo/src/grass7_trunk/gui/wxpython/dbmgr /osgeo/src/grass7_trunk/gui/wxpython/psmap /osgeo/src/grass7_trunk/gui/wxpython/iclass /osgeo/src/grass7_trunk/gui/wxpython/gmodeler /osgeo/src/grass7_trunk/man -- In case of errors please change into the directory with error and run 'make'. Please note that... If you get multiple errors, you need to deal with them in the order they appear in the error log. If you get an error building a library, you will also get errors from anything which uses the library. -- Finished compilation: Sun Jul 7 10:25:27 EEST 2013 make: *** [default] Error 1 ---%-- Running make inside some of the above mentioned directories, You should really start with the first error tells that some script was denied permissions to execute. I think it is a Gentoo/Funtoo thing and has nothing to do with GRASS or missing dependencies. For example, --%--- /osgeo/src/grass7_trunk/db/drivers/ogr $ make VERSION_NUMBER=7.0.svn /osgeo/src/grass7_trunk/dist.x86_64-unknown- linux-gnu/tools/g.html2man.py /osgeo/src/grass7_trunk/dist.x86_64- unknown-linux-gnu/docs/html/grass-ogr.html /osgeo/src/grass7_trunk/dist.x86_64-unknown-linux- gnu/docs/man/man1/grass-ogr.1 /bin/sh: /osgeo/src/grass7_trunk/dist.x86_64-unknown-linux- gnu/tools/g.html2man.py: Permission denied Check the permissions of /osgeo/src/grass7_trunk/dist.x86_64-unknown-linux-gnu/tools/g.html2man.py or --%--- nik@localhost /osgeo/src/grass7_trunk/gui/wxpython/vdigit $ make [..] /bin/sh: /osgeo/src/grass7_trunk/dist.x86_64-unknown-linux- gnu/scripts/g.gui.vdigit: Permission denied Check the permissions of /osgeo/src/grass7_trunk/dist.x86_64-unknown-linux-gnu/scripts/g.gui.vdigit Compilation should be done as normal user, not root. That means that the source should be obtained and copied to its current location as normal user. Maybe that helps. Markus M ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] i.segment on panchromatic band of Worldview 2 scene: what resources are necessary to complete segmentation ?
On Mon, Jul 8, 2013 at 2:18 PM, Moritz Lennert mlenn...@club.worldonline.be wrote: After some improvements to the module by MarkusM, I tried again during the night, but it got stuck again with heavy swapping going on. I'm definitively giving up to segment the panchromatic image in its entirety on my machine. Question: Does the memory= parameter limit total memory usage of the module, or does it only limit the memory usage for certain parts of it ? The memory= parameter should limit the total memory usage. In my tests, it used always a bit less memory than specified. It would be a bug if the module uses more memory than specified. The only limitations are 1) disk space, 2) how long you want to wait. Memory should not be a limit because of the memory option. On 07/07/13 11:52, Yann Chemin wrote: 2 - Cut in pieces (See Markus Comment about supercomputing) and run 4 quads on 4 CPUs instead of one large image in one CPU. I can do that. Will this have any influence on segmentation results ? Of course. There will be edge effects, objects will not cross region borders. Therefore you would need one final run with the pieces patched together, thus using the former output as input seeds for the last run. This last run should be faster and use less memory because it uses seeds. On a standard HPC system, you are usually allowed to compile your own software in your $HOME. You could compile trunk there, then set up the tiles and tile management, and launch jobs with the HPC's job manager (typically gridengine or derivates, torque and/or MPI). 3 - Recode i.segment to work natively on heterogeneous computers... (would be fun!) Would parallization help ? Should i.segment be included into the list of OpenMP candidates: http://grasswiki.osgeo.org/wiki/OpenMP#Candidates ? Hmm, because the final global run is needed to remove edge effects, I would expect the results to be different (not wrong, but different). But definitively an interesting idea! Markus M Moritz Good luck! On 4 July 2013 14:10, Moritz Lennertmlenn...@club.worldonline.be wrote: Hello, In parallel to the discussion going on in another thread, I have a question concering the segmentation of another Worldview 2 scene: I first used all 8 multispectral bands and managed to get a series of results with increasing thresholds in very reasonable running times. The region was as follows: g.region -p projection: 1 (UTM) zone: -36 datum: wgs84 ellipsoid: wgs84 north: 7251172 south: 7234772 west: 333792 east: 350192 nsres: 2 ewres: 2 rows: 8200 cols: 8200 cells: 6724 and the command line: i.segment group=xs out=seg_xs minsize=2 memory=3072 threshold=0.05 (and thresh=0.1 and 0.2 in successive runs using the results of the previous run as seeds). Now, I would like to test segmentation of just the panchromatic band. This means the following region settings: projection: 1 (UTM) zone: -36 datum: wgs84 ellipsoid: wgs84 north: 7251172 south: 7234772 west: 333792 east: 350192 nsres: 0.5 ewres: 0.5 rows: 32800 cols: 32800 cells: 107584 Trying to run with the following command line on my i3, 8GB RAM machine: i.segment group=pan out=seg_pan_005 threshold=0.05 memory=3072 had the process running for almost 13 hours with it then becoming apparently stuck in the fourth pass at 10%. At that point the percent didn't change for over an hour, so I decided to kill the process. Can I assume that I'm here above the capacities of my machine ? Is there anything (besides working on a smaller subsample of the image) that I can do to make it work ? What kind of resources would I need to be able to run such a segmentation? I guess I'll have to move these kinds of treatments to our university supercomputer, but I first have to get them to install GRASS... Moritz ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] GRASS 7 r.neighbours average method giving incorrect value
On Thu, Jul 11, 2013 at 1:30 PM, Glynn Clements gl...@gclements.plus.com wrote: Allar Haav wrote: I noticed today that average method in r.neighbours gives an incorrect value. Let me give you an example: 3) Judging from previous steps, it should be only logical that the output raster has values from 0 to 1. However it seems that 0.5 has been added: When the output is an integer map, the result of the average, variance and stddev methods is supposed to be rounded to the nearest integer (rather than towards zero), so 0.5 is added prior to truncation. For rounding of negative values, 0.5 would need to be subtracted, not added prior to truncation. See e.g. r.mapcalc's round(). Currently (in G6), the output map type is the same like the input map type. For an integer map with zeros and ones, the average will never be anything close to 0.5, but always either zero or one, not realistic and most probably not desired. I would rather have DCELL output all the times, maybe keep the maptype for min, max? Markus M ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] i.segment: invalid region id 0
On Fri, Jul 12, 2013 at 4:36 PM, Moritz Lennert mlenn...@club.worldonline.be wrote: Proxmox(Openvz) container running Debian testing, 200GB disk space, 10GB RAM, 4 i7 CPUs g.region -p projection: 1 (UTM) zone: 33 datum: wgs84 ellipsoid: wgs84 north: 4876400 south: 4849792 west: 610056 east: 634648 nsres: 0.5 ewres: 0.5 rows: 53216 cols: 49184 cells: 2617375744 r.univar 2013_04_24_orthorectified_mosaic_PAN_img02 100% total null and non-null cells: 2617375744 total null cells: 2061647833 Of the non-null cells: -- n: 555727911 i.group -l pan_r_ir group pan_r_ir references the following raster maps - 2013_04_24_orthorectified_mosaic_PAN_img02@PERMANENT 2013_04_24_orthorectified_mosaic_XS_img02.5@PERMANENT 2013_04_24_orthorectified_mosaic_XS_img02.7@PERMANENT time i.segment -w group=pan_r_ir out=seg_weighted_pan_r_ir threshold=0.01 memory=8192 Loading input bands... Pass 1: ERROR: Invalid region id 0 Please try r57074. Markus M I still have over 100GB of disk space available. What could be the problem ? Moritz ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] GRASS GIS 7 tech-preview release preparations
On Fri, Jul 12, 2013 at 3:21 PM, Moritz Lennert mlenn...@club.worldonline.be wrote: Main issues I see for grass7 currently to get it preview-release ready: [...] - LFS handling in Windows https://trac.osgeo.org/grass/ticket/1131 (not sure I understand where we are at with this) Global LFS is enabled by default and working on all officially supported platforms, plus *BSD and some UNIX systems. BTW: http://www.heise.de/open/artikel/Die-Woche-1-fuer-schnelle-Veroeffentlichungen-1916127.html -- new release-often cycle of Firefox positively discussed I think that once 6.4.3 and the grass7 preview-release are out, we should work on a more structured release process. At least for grass7 tech previews, it might be worth thinking about a release early, release often strategy, while clearly declaring these release as what they are, i.e. without absolute guarantee of stability. +1 Markus M Moritz ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] GRASS GIS 7 tech-preview release preparations
On Sat, Jul 13, 2013 at 12:16 PM, Moritz Lennert mlenn...@club.worldonline.be wrote: On Fri, July 12, 2013 22:04, Markus Metz wrote: On Fri, Jul 12, 2013 at 3:21 PM, Moritz Lennert mlenn...@club.worldonline.be wrote: Main issues I see for grass7 currently to get it preview-release ready: [...] - LFS handling in Windows https://trac.osgeo.org/grass/ticket/1131 (not sure I understand where we are at with this) Global LFS is enabled by default and working on all officially supported platforms, plus *BSD and some UNIX systems. That means we can close this ticket ? Yes, done. Markus M ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] GRASS GIS 7 tech-preview release preparations
On Sat, Jul 13, 2013 at 11:47 PM, Hamish hamis...@yahoo.com wrote: - LFS handling in Windows https://trac.osgeo.org/grass/ticket/1131 MarkusM: Global LFS is enabled by default and working on all officially supported platforms, plus *BSD and some UNIX systems. Moritz: That means we can close this ticket ? it depends: is LFS working in MS Windows? i.e. has it been tested with the latest trunk code and passed? We shouldn't close tickets on assumptions. Yes, I tested on Windows and Linux before committing the latest LFS-related changes. Markus M thanks, Hamish ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
[GRASS-dev] mapset permissions: only owner should have write permissions
From within GRASS, only the owner of a mapset is allowed to start a GRASS session in this mapset, i.e. only the owner of a mapset has write permissions to this mapset. But a new mapset being a folder in the file system is created with mode 0777, thus granting write permissions to all. I suggest to change mode from 0777 to 0755 in G_mkdir() and add mode = 0755 in gis_set.py. Any objections? Markus M ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] mapset permissions: only owner should have write permissions
On Sun, Jul 14, 2013 at 10:42 PM, Hamish hamis...@yahoo.com wrote: Hi, Markus M: From within GRASS, only the owner of a mapset is allowed to start a GRASS session in this mapset, i.e. only the owner of a mapset has write permissions to this mapset. But a new mapset being a folder in the file system is created with mode 0777, thus granting write permissions to all. I suggest to change mode from 0777 to 0755 in G_mkdir() and add mode = 0755 in gis_set.py. Any objections? It's not created as 777 (I would have noticed that :), that's before the umask. So for me it gets created as drwxr-xr-x, as expected. For me, it gets created as drwxr-xr-x too, but this depends on umask. G_mkdir() calls (on anything but windows) mkdir(path, 0777) [0] $ man 2 mkdir: The argument mode specifies the permissions to use. It is modified by the process's umask in the usual way: the permissions of the created directory are (mode ~umask 0777). Other mode bits of the created directory depend on the operating system. man 2 umask: The typical default value for the process umask is S_IWGRP | S_IWOTH (octal 022) which results in the desired drwxr-xr-x permissions. I could not find information on typical, and this typical might be different on different systems. I suggest to make sure that the permissions are really drwxr-xr-x by using mkdir(path, 0755);. Markus M [0] https://trac.osgeo.org/grass/browser/grass/trunk/lib/gis/paths.c#L32 [...] NOTES Under Linux apart from the permission bits, only the S_ISVTX mode bit is honored. That is, under Linux the created directory actually gets mode (mode ~umask 01777). See also stat(2). Hamish ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] i.segment: invalid region id 0
On Mon, Jul 15, 2013 at 10:04 AM, Moritz Lennert mlenn...@club.worldonline.be wrote: On Fri, July 12, 2013 21:59, Markus Metz wrote: On Fri, Jul 12, 2013 at 4:36 PM, Moritz Lennert mlenn...@club.worldonline.be wrote: Proxmox(Openvz) container running Debian testing, 200GB disk space, 10GB RAM, 4 i7 CPUs g.region -p projection: 1 (UTM) zone: 33 datum: wgs84 ellipsoid: wgs84 north: 4876400 south: 4849792 west: 610056 east: 634648 nsres: 0.5 ewres: 0.5 rows: 53216 cols: 49184 cells: 2617375744 r.univar 2013_04_24_orthorectified_mosaic_PAN_img02 100% total null and non-null cells: 2617375744 total null cells: 2061647833 Of the non-null cells: -- n: 555727911 i.group -l pan_r_ir group pan_r_ir references the following raster maps - 2013_04_24_orthorectified_mosaic_PAN_img02@PERMANENT 2013_04_24_orthorectified_mosaic_XS_img02.5@PERMANENT 2013_04_24_orthorectified_mosaic_XS_img02.7@PERMANENT time i.segment -w group=pan_r_ir out=seg_weighted_pan_r_ir threshold=0.01 memory=8192 Loading input bands... Pass 1: ERROR: Invalid region id 0 Please try r57074. I still get exactly the same error. Anything I can do to debug ? You will need to add loops that go through the segment ids and check if any id is set to zero. Another possibility is that the region growing algorithm does not find a best neighbor for a given region, which it should. There should always be a valid best neighbor. I can prepare a patch for you with additional debug tests, hopefully until this afternoon, then you could have a chance to start the module again before you go offline. Markus M Moritz P.S. I'll be offline between this afternoon and Thursday morning. ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] mapset permissions: only owner should have write permissions
On Mon, Jul 15, 2013 at 5:55 PM, Glynn Clements gl...@gclements.plus.com wrote: Markus Metz wrote: From within GRASS, only the owner of a mapset is allowed to start a GRASS session in this mapset, i.e. only the owner of a mapset has write permissions to this mapset. But a new mapset being a folder in the file system is created with mode 0777, thus granting write permissions to all. I suggest to change mode from 0777 to 0755 in G_mkdir() and add mode = 0755 in gis_set.py. Any objections? I don't see why GRASS should be special in this regard. The convention is that programs should allow the user to control read and write permissions via the umask, while execute permission is determined by the program. In this case, would it be ok to enforce umask to 0022 in the start up script? Programs creating files containing particularly-sensitive information (e.g. encryption keys) may reasonably impose more restrictive permissions. With grass data on a network drive with multi-user access, I would regard e.g. the contents of the PERMANENT mapset as particularly-sensitive information. GRASS already includes its own ownership check to prevent users from shooting each other in the foot with shared directories (by creating subdirectories which the owner cannot remove). So I don't really see any reason to enforce the policy a second time through filesystem permissions. An inexperienced user trying to make a backup of a grass dataset, syncing the wrong way... Anyway, I withdraw my suggestion to use 0755 as default mode for mkdir(path, mode). It's probably safer to enforce the system's default mask directly. Markus M ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] i.segment: invalid region id 0
On Mon, Jul 15, 2013 at 11:09 PM, Nikos Alexandris n...@nikosalexandris.net wrote: Moritz Lennert wrote: [..] time i.segment -w group=pan_r_ir out=seg_weighted_pan_r_ir threshold=0.01 memory=8192 Loading input bands... Pass 1: ERROR: Invalid region id 0 [..] What could be the problem ? Moritz, Markus, the same error today while performing a random test. Disk capacity and RAM plenty here. time i.segment group=pan out=pan_segmented_plantations_area_threshold_1 threshold=0.99 In most cases, this threshold is too large. It should be between 0 and 1, and rather close to zero. [...] Pass 13: ERROR: Invalid region id 0 real40m44.339s user37m34.357s sys 0m0.380s Details projection: 1 (UTM) zone: 38 datum: wgs84 ellipsoid: wgs84 north: -2571782.5 south: -2574924.5 west: 359153.5 east: 359525 nsres: 0.5 ewres: 0.5 rows: 6284 cols: 743 cells: 4669012 Let me know how I can help. Can you make the dataset available (off-list if need be)? I can not reproduce this error with the datasets I have. Markus M ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] standardized options
You could try G_OPT_M_DIR. The comment says directory input but it should also work for output, all it should return is the path to a folder. Markus M On Mon, Jul 15, 2013 at 5:42 PM, Margherita Di Leo dileomargher...@gmail.com wrote: Dear Devs, in r57084 Martin has added standardized options to r.ipso (thanks!). I would like to learn more about their use, particularly, does it exist a standard variable defined for taking a path as input? (Not a path+file name as G_OPT_F_INPUT). For example this is useful in the case that a module produces some pictures and/or csv files and the user wants to set a folder for storing them. I digged into [1] but I seem not to find what I need. Should I just take the path as a string? Giving to the user the possibility to browse the folders instead of indicating a path would be IMHO more convenient. Could anyone point me out where to read in order to learn more about that? Thanks! madi [1] http://grass.osgeo.org/programming7/gis_8h_source.html -- Best regards, Dr. Margherita DI LEO Scientific / technical project officer European Commission - DG JRC Institute for Environment and Sustainability (IES) Via Fermi, 2749 I-21027 Ispra (VA) - Italy - TP 261 Tel. +39 0332 78 3600 margherita.di-...@jrc.ec.europa.eu Disclaimer: The views expressed are purely those of the writer and may not in any circumstance be regarded as stating an official position of the European Commission. ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] mapset permissions: only owner should have write permissions
On Tue, Jul 16, 2013 at 10:38 PM, Hamish hamis...@yahoo.com wrote: Markus Metz wrote: In this case, would it be ok to enforce umask to 0022 in the start up script? two rules to thumb: we shouldn't restrict others to the limits of our own imagination. (if someone wants to set their umask to allow 777 for whatever reason or need, why not let them?) it's not broken == don't fix it ... my suggestion is just leave the thing alone. why make work for people? Just to clarify, I no longer want to enforce permissions inside GRASS, but will instead enforce permissions on system level, outside GRASS. Maybe our shared network GRASS data are a special case. Markus M ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
[GRASS-dev] r57119 v.pack: move db and proj file to the root, preparation for packaging multiple maps
Please update v.unpack accordingly. Thanks, Markus M ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] r57119 v.pack: move db and proj file to the root, preparation for packaging multiple maps
On Tue, Jul 23, 2013 at 11:10 PM, Martin Landa landa.mar...@gmail.com wrote: Hi, 2013/7/23 Markus Metz markus.metz.gisw...@gmail.com: Please update v.unpack accordingly. I checked `v.unpack`. What exactly is not working? I haven't found any problem related to db [1] and proj file [2]. Sorry for the confusion, I received a vector packed with a pre-r57119 v.pack which the current v.unpack could not unpack. Markus M Martin [1] http://trac.osgeo.org/grass/browser/grass/trunk/scripts/v.unpack/v.unpack.py#L109 [2] http://trac.osgeo.org/grass/browser/grass/trunk/scripts/v.unpack/v.unpack.py#L101 -- Martin Landa landa.martin gmail.com * http://geo.fsv.cvut.cz/~landa ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] Right way to use G_getl2
I think the confusion arises because G_getl2(char *buf, int n, FILE *
fd) reads in at most n - 1 characters, not n characters. The next
character is always set to '\0' and guaranteed to be at most the n-th
character. That also means that G_getl() does not check if the buffer
is large enough to hold the line to be read, which it should, IMHO.
To the example with v.in.ascii:
The module first finds the longest line, the length including the
terminating character. This number is then passed to G_getl2() as n,
which means that the end of the line is not read for the longest line,
because the end of the line is the n-th character.
I would suggest to modify G_getl2() to read at most n (not n - 1)
characters and include a check if the buffer is long enough:
---
Index: getl.c
===
--- getl.c(revision 57537)
+++ getl.c(working copy)
@@ -67,7 +67,7 @@
int c;
int ret = 1;
-while (i n - 1) {
+while (i n) {
c = fgetc(fd);
if (c == EOF) {
@@ -88,6 +88,11 @@
}
break;
}
+
+if (i == n - 1) {
+G_warning(G_getl2(): buffer too small);
+break;
+}
buf[i] = c;
---
Markus M
On Fri, Sep 13, 2013 at 4:58 AM, Vaclav Petras wenzesl...@gmail.com wrote:
Hi,
this relates to the recent fix for r.in.ascii (r57649). The skip parameter
[2] required one line more because when reading lines using G_getl2 [3]
function and counting these lines it counted one line more.
The documentation of the function says that it reads max n-1 bytes. In other
words, n is the size of the buffer, so it is clear that it can n-1 useful
characters plus string terminating character '\0'. Quite standard way but...
The problem in v.in.ascii was that it the caller passed the previously
determined maximum number of characters on line in file plus one (space for
the terminating character). So this number is the same for all lines and it
works well for all lines except for the longest one.
The reason is that G_getl2 stops reading characters for the longest line and
leaves end line character in the stream. So, in fact, the longest line is in
buffer and can be processed. However, the next line is than only the
remaining end line character. This is not what the caller expected. There is
even no way to find out that the line was not read completely.
So, finally: What is the right usage of G_getl2? Should the caller use
buffer size equal to maximum number of expected characters on line plus end
of line character (plus 2 on MS Windows) plus terminating character? Or
should he just pass the n parameter increased by one (two on MS Windows)
since in fact nothing will be stored to buffer? Or should he just allocate
really huge buffer and than read chars from buffer if he wants to store
them?
Thanks,
Vaclav
[1] https://trac.osgeo.org/grass/changeset/57649
[2] http://grass.osgeo.org/grass70/manuals/v.in.ascii.html
[3]
http://grass.osgeo.org/programming7/getl_8c.html#a4bf86e666fda18059c42b0f5eca6f9bd
Note 1: Don't be confused from documentation since the n parameter
description for G_getl2 is wrong.
Note 2: In description I was taking only about Unix line endings. On Windows
we need two bytes more to store it as noted in questions. The case of old
Mac OS is not cover because it is even more tricky in G_getl2 function.
___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev
___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] v.neighbors in GRASS7 much slower ?
On Fri, Sep 13, 2013 at 5:02 PM, Moritz Lennert mlenn...@club.worldonline.be wrote: Hi, In grass64release, running: v.distance from=hospitals@PERMANENT to=hospitals@PERMANENT col=to_cat,dist up=cat,dist -p dmin=0.0001 gives me an almost instant response. In grass7, it takes _much_ longer. What has changed ? That was a bug in the new search method, fixed in r57657. Markus M ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] Right way to use G_getl2
Glynn Clements wrote: FWIW, I suggest changing the existing behaviour [of G_getl2()] so that it always reads an entire line, even if it can't store it all. IOW, read until EOL or EOF, stop storing characters once the buffer is full. +1 Unlike with fgets(), which stores the line terminator in the buffer, there is no way for the caller of G_getl2() to determine that an incomplete line was read. If the buffer is too small, you lose either way; discarding the trailing characters is likely to be more robust than returning them as if they were a subsequent line (the current behaviour). The caller could know if a line was only partially read if G_getl2() would return -1 (anything but 0, 1) in case the buffer is too small. Markus M ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] How to calculate mean coordinates from big point datasets?
On Wed, Sep 18, 2013 at 11:41 AM, Moritz Lennert
mlenn...@club.worldonline.be wrote:
On 18/09/13 10:51, Luca Delucchi wrote:
On 17 September 2013 22:10, Markus Netelernete...@osgeo.org wrote:
Hi,
I came across this question:
http://gis.stackexchange.com/questions/71734/how-to-calculate-mean-coordinates-from-big-point-datasets
and wondered if this approach would be the fasted:
# http://grass.osgeo.org/sampledata/north_carolina/points.las
v.in.lidar input=points.las output=lidarpoints -o
...
Number of points: 1287775
...
Now I would use
v.univar -d lidarpoints type=point
(still calculating here...)
Is it the best way?
maybe v.median [0] could help?
[1]
http://trac.osgeo.org/grass/browser/grass-addons/grass7/vector/v.median
Right.
Here's a little test:
$time v.median in=elev_lid792_randpts
638648.50|220378.50
Should be 638648|220378. It seems that numpy gets the median wrong...
real0m0.249s
user0m0.180s
sys 0m0.044s
$time v.to.db elev_lid792_randpts op=coor -p | awk -F'|' 'BEGIN{SUMX=0;
SUMY=0; N=0} {N+=1;SUMX+=$2;SUMY+=$3} END{print SUMX/N, SUMY/N}'
Reading features...
100%
638544 220339
Should be 638650 220376
real0m0.106s
user0m0.100s
sys 0m0.020s
Would be interesting to see results for big data. And AFAIK median is a bit
more difficult to do in awk. I imagine that replacing the median by the mean
in numpy is no problem (might be a flag to add to v.median).
The v.to.db + v.db.univar approach is working just fine, and provides
correct results.
About a little module to calculate centroid of polygon, center point
of line and centroid (possibly weighted) for points, that would be
easy because all the components are there, even though there are in
theory alternatives to the current calculation of centroids for
polygons.
Markus M
___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] How to calculate mean coordinates from big point datasets?
On Thu, Sep 19, 2013 at 9:15 AM, Moritz Lennert mlenn...@club.worldonline.be wrote: On 18/09/13 16:24, Markus Metz wrote: Moritz Lennert wrote: Here's a little test: $time v.median in=elev_lid792_randpts 638648.50|220378.50 Should be 638648|220378. It seems that numpy gets the median wrong... When I look at the numbers coming out of v.to.db, there are a series of points at X=638648,5 around the 3000 limit, and a series of points at Y=220378,5, so I do believe that numpy is right. Right, db.univar was printing out with insufficient precision, fixed in r57750. About a little module to calculate centroid of polygon, center point of line and centroid (possibly weighted) for points, that would be easy because all the components are there, even though there are in theory alternatives to the current calculation of centroids for polygons. And these alternatives are better ? I am not sure. For areas without isles, there is a faster alternative. For areas with isles, the current approach is fast, but the centroids might be placed at somewhat unexpected locations (the next best location inside the area, outside its isles). Here, alternatives would be much slower. Markus M ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] How to calculate mean coordinates from big point datasets?
Glynn Clements wrote: Luca Delucchi wrote: maybe v.median [0] could help? Not for large datasets. First, it requires that the data will fit into RAM. Second, numpy.median() sorts the entire array and takes the middle value, which is somewhere between O(n.log(n)) for the typical case and O(n^2) for the worst case (numpy.median uses the default sorting algorithm, which is quicksort). See r.quantile for a more efficient approach for large datasets. The first pass computes a histogram which allows upper and lower bounds to be determined for each quantile. The second pass extracts values which lie within those bounds and sorts them. The approach of v.median and r.quantile regards each dimension separately which is IMHO not correct in a spatial context. The median of a point cloud would be a multivariate median for 2 or 3 dimensions. You would need to sort the points first by the first dimension, then by the second dimension etc, then pick the coordinates of the mid point. Alternatively you would need to find the point with the smallest distance to all other points, which is nightmare to calculate ( (n - 1) x (n - 1) distance calculations). Except for pathological cases (where the majority of the data lie within a tiny proportion of the overall range), only a small fraction of the data are sorted. Coordinates of large point clouds can easily be such pathological cases. In any case: is this question about the mean or the median? I guess the median could make sense to account for outliers. A v.centerpoint module could thus have options for point mode (mean, median, distance) and line mode (mid point, center of gravity). Area centroids are already calculated by v.category type=area, even though they are not centers of gravity. Markus M ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] How to calculate mean coordinates from big point datasets?
On Fri, Sep 20, 2013 at 5:38 PM, Markus Metz markus.metz.gisw...@gmail.com wrote: Glynn Clements wrote: Luca Delucchi wrote: maybe v.median [0] could help? Not for large datasets. First, it requires that the data will fit into RAM. Second, numpy.median() sorts the entire array and takes the middle value, which is somewhere between O(n.log(n)) for the typical case and O(n^2) for the worst case (numpy.median uses the default sorting algorithm, which is quicksort). See r.quantile for a more efficient approach for large datasets. The first pass computes a histogram which allows upper and lower bounds to be determined for each quantile. The second pass extracts values which lie within those bounds and sorts them. The approach of v.median and r.quantile regards each dimension separately which is IMHO not correct in a spatial context. This The median of a point cloud would be a multivariate median for 2 or 3 dimensions. You would need to sort the points first by the first dimension, then by the second dimension etc, then pick the coordinates of the mid point. is wrong, please ignore. This Alternatively you would need to find the point with the smallest distance to all other points, which is nightmare to calculate ( (n - 1) x (n - 1) distance calculations). is correct, but instead of finding the existing point with the smallest distance, that point can be approximated with less effort. Markus M ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] How to calculate mean coordinates from big point datasets?
Markus Neteler wrote: Hi, I came across this question: http://gis.stackexchange.com/questions/71734/how-to-calculate-mean-coordinates-from-big-point-datasets Please try the new addon v.centerpoint [0]. It calculates various center points for point clouds, lines, and areas. Standard options are the geometric mean (center of gravity) and geometric median (more robust for outliers). There are additional options to calculate center points for points, lines and areas, explained in the manual. The geometric medians (points of minimum distance to all other points) are approximations, but fairly accurate and very fast. I would be surprised if any other GIS software can calculate these center points. Markus M [0] http://grasswiki.osgeo.org/wiki/AddOns/GRASS7/vector#v.centerpoint ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] v.kcv enahncement- was Re: How to calculate mean coordinates from big point datasets?
Helena Mitasova wrote: Markus, when you are at it, can you also have a look at v.kcv? http://grass.osgeo.org/grass70/manuals/v.kcv.html I am wondering whether it works efficiently with lidar data sets (millions of points) - I heard that it takes forever, but that was about a year ago and I haven't tried it myself. v.kcv has been improved recently in trunk, thanks to Jan Vandrol and Jan Ruzicka. For example, if I want to partition the data set into 1% test points and 99% given data points (for example to test interpolation) it appears I may need 100 partitions as there is no way to have just two partitions with different size. The number of partitions does not influence speed (in trunk). One of the problems may be the table - perhaps if this was run without the table and the output was written into two (or k) separate files, it could be faster? Yes, updating the table can be slow. For a large number of points it is recommended to not use dbf. Creating a separate new vector for each partition could be an alternative. The core of the algorithm which is based on finding the closest points to the set of random points should allow this. This is the part that makes v.kcv slow in 6.x. Creating a subset of points which are farther apart than given threshold (2d or 3d distance) would be also useful (it is done internally in v.surf.rst and I have a version which does it with 3D distances, but the resulting subset is not written into output file). For that you would need a spatial search structure in order to be reasonably fast. I guess that v.surf.rst uses quad- or oct-trees for this purpose. This is not urgent but if it is not too difficult it would be nice to have, or let us know if it already works and I just cannot find the right module, As of 2013-07-19, v.kcv in trunk is much faster than in 6.x. Creating subsets of points which are farther apart than given threshold is not implemented, but that would not be too difficult to add using a spatial index for each partition. Markus M ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] How to calculate mean coordinates from big point datasets?
Glynn Clements wrote: Markus Metz wrote: Please try the new addon v.centerpoint [0]. It calculates various center points for point clouds, lines, and areas. Standard options are the geometric mean (center of gravity) That's the arithmetic mean. The geometric mean of a set of N values is the Nth root of the product of the values (i.e. the logarithm of the geometric mean is the arithmetic mean of the logarithms of the values). It wouldn't be meaningful to compute the geometric mean of coordinate data. Right, I fixed the documentation. Internally, the arithmetic mean has been used already to calculate centers of gravity. Markus M ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] compile error with v.in.ply
Paulo van Breugel wrote: I am trying to compile GRASS GIS 7.0 (revision 57836) and I am getting an error when running make related to v.in.ply and v.out.ply. When running make in the vector folder, I am getting the following error message: make: *** v.in.ply: No such file or directory. Stop. make: *** v.out.ply: No such file or directory. Stop. Oops, fixed in r57837. Markus M ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] v.in.ogr: how to force sqlite instead of dbf for output location ? SEG-Y input (Geology/Seismic Data)
Peter Löwe wrote: When trying to ingest SEG-Y data (geology/seismics) into a new location via v.in.ogr (GDAL 1.9.0) this error is thrown since the new location is set by default (?) to dbf: [...] The initial location from which v.in.ogr was invoked was already switched to a sqlite-backend. How can this issue be overcome ? You can use from any location g.proj georef=demo20071121142735_CH1_35P.seg location=segy_sqlite This will create a new location with the projection information of the given dataset, but will not import the dataset. Then you would need to switch to a mapset in the new location, set default db backend to sqlite with db.connect and import the dataset with v.in.ogr. Or easier, use GRASS 7. HTH, Markus M ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] GRASS 6.4.svn on AIX based supercomputers
Have you included -R,/opt/freeware/lib in LDFLAGS? If not, you probably should. See also GRASS wiki [0], there the entry for AIX 7.x. Markus M [0] http://grasswiki.osgeo.org/wiki/Compile_and_Install#AIX On Mon, Sep 30, 2013 at 9:35 PM, Markus Neteler nete...@osgeo.org wrote: Hi, I backported the relevant things from trunk, now GRASS 6.4.svn (upcoming 6.4.4) compiles on AIX powerpc systems, too! While this is fine: GRASS 6.4.4svn (patUTM32):~/software/grass-6.4.svn g.version -r GRASS 6.4.4svn (2013) Revision: 50937 Date: 2012-02-25 14:14:51 +0100 (Sat, 25 Feb 2012) I struggle here (same error as for GRASS 7 on the same system): GRASS 6.4.4svn (patUTM32):~/software/grass-6.4.svn g.list vect Could not load program g.list: Dependent module libstdc++.a(libstdc++.so.6) could not be loaded. Could not load module libstdc++.a(libstdc++.so.6). System error: No such file or directory GRASS 6.4.4svn (patUTM32):~/software/grass-6.4.svn which g.list /gpfs/home/neteler/software/grass-6.4.svn/dist.powerpc-ibm-aix7.1.0.0/bin/g.list GRASS 6.4.4svn (patUTM32):~/software/grass-6.4.svn ldd /gpfs/home/neteler/software/grass-6.4.svn/dist.powerpc-ibm-aix7.1.0.0/bin/g.list /gpfs/home/neteler/software/grass-6.4.svn/dist.powerpc-ibm-aix7.1.0.0/bin/g.list needs: /gpfs/home/neteler/software/grass-6.4.svn/dist.powerpc-ibm-aix7.1.0.0/lib/libgrass_vect.6.4.4svn.so /gpfs/home/neteler/software/grass-6.4.svn/dist.powerpc-ibm-aix7.1.0.0/lib/libgrass_dbmibase.6.4.4svn.so /gpfs/home/neteler/software/grass-6.4.svn/dist.powerpc-ibm-aix7.1.0.0/lib/libgrass_gis.6.4.4svn.so /gpfs/home/neteler/software/grass-6.4.svn/dist.powerpc-ibm-aix7.1.0.0/lib/libgrass_datetime.6.4.4svn.so /usr/lib/libz.a(libz.so.1) /gpfs/home/neteler/software/grass-6.4.svn/dist.powerpc-ibm-aix7.1.0.0/lib/libgrass_dbmiclient.6.4.4svn.so /gpfs/home/neteler/software/grass-6.4.svn/dist.powerpc-ibm-aix7.1.0.0/lib/libgrass_dgl.6.4.4svn.so /gpfs/home/neteler/software/grass-6.4.svn/dist.powerpc-ibm-aix7.1.0.0/lib/libgrass_dig2.6.4.4svn.so /gpfs/home/neteler/software/grass-6.4.svn/dist.powerpc-ibm-aix7.1.0.0/lib/libgrass_rtree.6.4.4svn.so /gpfs/home/neteler/software/grass-6.4.svn/dist.powerpc-ibm-aix7.1.0.0/lib/libgrass_linkm.6.4.4svn.so Cannot find libstdc++.a(libstdc++.so.6) /usr/lib/libdl.a(shr_64.o) /usr/lib/libiconv.a(libiconv.so.2) ar: 0707-109 Member name libiconv.so.2 does not exist. dump: /tmp/tmpdir4325472/extract/libiconv.so.2: 0654-106 Cannot open the specified file. /gpfs/home/neteler/software/grass-6.4.svn/dist.powerpc-ibm-aix7.1.0.0/lib/libgrass_g3d.6.4.4svn.so /usr/lib/libc.a(shr_64.o) Cannot find libstdc++.a(libstdc++.so.6) Cannot find libgcc_s.a(shr.o) /usr/lib/libc.a(shr.o) /unix /usr/lib/libcrypt.a(shr_64.o) /usr/lib/libcrypt.a(shr.o) GRASS 6.4.4svn (patUTM32):~/software/grass-6.4.svn ls -la /opt/freeware/lib64/ | grep std GRASS 6.4.4svn (patUTM32):~/software/grass-6.4.svn ls -la /opt/freeware/lib/ | grep std lrwxrwxrwx1 root system 44 Aug 27 19:24 libstdc++.a - gcc/powerpc-ibm-aix7.1.0.0/4.6.1/libstdc++.a Any pointers? thanks Markus ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] creation of turntable
Moritz Lennert wrote: On 30/09/13 09:45, Štěpán Turek wrote: Hi, working on integration of turns support into GRASS 7 I see two possibilities how to integrate creation of the turntable. New module v.net.turntable can be added,which will manage creation of the turntable. There already exists v.net module which is doing maintenance and preparation of a network for a network analysis. Creation of the turntable can be considered also like preparation of network for analysis. Also in future it would be good to add into connect operation support for modification of the turntable. And this will require extension of the v.net interface for the turntable support. Suggestion of interface changes of v.net module: operation=turntable - create turntable (what v.net.turntable do) tlayer and tuclayer - two layers where information about turns is stored (for description see [1]) - these two parameters would need to be added into v.net even if v.net.turntable would exist in order to be the connect operation able to adjust the turntable. Adding of these parameters could be avoided if another connect method would be implemented into v.net.turntable, but it would be probably quiet confusing for user. -t flag if this flag would be attached, the connect method would work with the turntable support On the other hand v.net module interface will become even more complex and it can become difficult to use by some users. Even if the complexity might be increased (I don't find v.net _that_ complex at this point), I would go for the integration of turntables in v.net. It seems the logical place for everything concerning network preparation and maintenance. If complexity really becomes a problem, there is always the option of creating wrapper scripts which hide some of the complexity. Sounds good to me. Regarding the GUI, all options/flags relevant for turntables could go into a separate GUI section. Markus M ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] r.in.lidar tuning
Markus Neteler wrote: PS: if the LAS file is not accessible, r.in.lidar happily segfaults. My attempts to fix that were yet unsuccessful. Fixed in r57951,2 for [r|v].in.lidar Markus M ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] r.in.lidar tuning
Markus Neteler wrote: One more wish comes to mind (indeed, I started cross-porting but :-) v.in.lidar comes with a filter (which should perhaps be generalized to the nth return; or, the last *is* the nth return but then mid could be more than one in this case?): Yes, for n returns, the first return is number 1, the last return is the nth return and mid returns are all returns in between, i.e. none for 2 returns. Choosing the nth return does not make sense to me. filter Only import points of selected return type If not specified, all points are imported options: first,last,mid which would be great for r.in.lidar as well to avoid that I need to split the file with las2las beforehand. Why would you want to do filtering? The r.in.lidar methods min, max, mean, median, percentile are not sufficient? Markus M ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] i.segment fails
Javier Martínez-López wrote: Dear list, I am trying to run the i.segment function but I always get the same error during the first pass: 'invalid region id 0'. What does it mean? can it be solved? I have read some e-mails about this problem but have found no solution yet. Trying with different thresholds or changing the memory limit does not work. My output: GRASS 7.0.svn (global_MW):/home/user/HRI/Variables time i.segment gr=segm out=virunga_segm_0.5 thre=0.5 mins=15 memory=5 Pass 1: ERROR: Invalid region id 0 real0m2.364s user0m2.264s sys0m0.094s # version=7.0.svn date=2013 revision=55075 i.segment was last changed in r57267, which should also fix the error observed by you. Please update to r57267 or higher and try again. i.segment should now finish successfully. If not, please let us know and if possible provide test data and relevant settings. Markus M ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
[GRASS-dev] WANTED: 64bit big endian test system
We would like to make GRASS GIS portable to 32 bit and 64 bit systems as well as little endian and big endian systems. To our knowledge, all combinations of bitness and endianness are supported, with the exception of 64bit big endian test systems. Therefore we need access to a 64 bit big endian test system with an operating system with a 64bit userland [0] including a C compiler in order to perform testing and bug fixing. We are aware of the fact that such systems are rather institutional systems (supercomputers, mainframes, High Performance Clusters) than office or personal devices. Please contact any of the GRASS developers off-list if you are able and willing to facilitate access to a 64bit big endian system. Thanks, Markus M [0] https://wiki.debian.org/Sparc64#Rationale_for_such_a_port PS: Unfortunately, AIX as OS is unsuitable because AIX deviated too far from *NIX and POSIX standards. ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] test of skeleton functionality in v.voronoi
Moritz Lennert wrote: Markus M, I know this is still very fresh, but I was excited about seeing your addition to v.voronoi allowing to calculate skeletons and longest lines for areas. Amongst others, this could be useful for calculating some shape parameters of polygons for region-based classification. So, just a quick feedback. I ran the module on the entire urbanarea map, in order to test scalability. The results are really nice and I haven't found evident errors, yet. I found evident errors in some other test data. The voronoi algorithm suffers from problems related to floating point representation errors, also sometimes apparent when calculating voronoi diagrams for areas. It takes quite a while, though: Yes, because it is a brute force approach. The problem of finding the center line for areas, or an area skeleton from which the center line can be extracted, must have been solved previously. A literature research might help. On first glance, I did not find a solution, though. Only for straight skeletons which are not suitable to extract a center line. Markus M ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] new flag to interpolate from nearby raster cells for v.what.rast
On Thu, Nov 14, 2013 at 11:15 AM, Hamish hamis...@yahoo.com wrote: Hi, I just added in the dev branches two new flags for v.what.rast. [...] The second flag changes the default containing-grid-cell method to a weighted avg. of the 4 nearest raster cell centers (IDW). The search radius is not very big [...] AFAIK, IDW takes as argument the radius which is fixed to 1 which is questionable. Since v.what.rast samples a raster value at a given location, why not use the standard raster sampling methods nearest: nearest neighbor linear: linear interpolation cubic: cubic convolution lanczos: lanczos filter linear_f: linear interpolation with fallback cubic_f: cubic convolution with fallback lanczos_f: lanczos filter with fallback which are used by r.proj and i.rectify? The concept is essentially the same: for a given location, estimate the corresponding raster value. Markus M ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] compilation of grass on AIX 7.1
if some libraries are in a non-standard location, you might need to add -Wl,-bsvr4,-R,/opt/freeware/lib or equivalent to LDFLAGS [0]. Markus M [0] http://grasswiki.osgeo.org/wiki/Compile_and_Install#AIX On Sun, Nov 24, 2013 at 8:16 PM, Markus Neteler nete...@osgeo.org wrote: On Sun, Nov 24, 2013 at 5:07 PM, Glynn Clements gl...@gclements.plus.com wrote: Markus Neteler wrote: Any hints how to inject libstdc++.so for those modules which depend on $(GPROJLIB)? Maybe a trivial issue but I just don't know what to change. AFAICT, libgrass_gproj only requires libstdc++ because of GDAL. Yes, I see. So the definition of GDALLIBS in Platform.make should include -lstdc++ (and anything else which is required for C++ support). Unfortunately adding it: svn diff include/ Index: include/Make/Platform.make.in === --- include/Make/Platform.make.in (revision 58291) +++ include/Make/Platform.make.in (working copy) @@ -157,7 +157,7 @@ LAPACKINC = @LAPACKINC@ #GDAL/OGR -GDALLIBS= @GDAL_LIBS@ +GDALLIBS= @GDAL_LIBS@ -lstdc++ GDALCFLAGS = @GDAL_CFLAGS@ USE_GDAL= @USE_GDAL@ USE_OGR = @USE_OGR@ ... does not help. I checked, it is present in include/Make/Platform.make However, I don't understand why this would only affect libgrass_gproj and not other libraries which require GDAL. Indeed it does, I looped over it now. Here the reduced list of failure (the others ldd checks report no error): libgrass_cluster.7.0.svn.so needs: /gpfs/home/neteler/software/grass-7.0.svn/dist.powerpc-ibm-aix7.1.0.0/lib/libgrass_imagery.7.0.svn.so /gpfs/home/neteler/software/grass-7.0.svn/dist.powerpc-ibm-aix7.1.0.0/lib/libgrass_raster.7.0.svn.so /gpfs/home/neteler/software/grass-7.0.svn/dist.powerpc-ibm-aix7.1.0.0/lib/libgrass_gis.7.0.svn.so /usr/lib/libc.a(shr_64.o) /usr/lib/libpthread.a(shr_xpg5_64.o) /gpfs/home/neteler/software/grass-7.0.svn/dist.powerpc-ibm-aix7.1.0.0/lib/libgrass_vector.7.0.svn.so /opt/freeware/lib/libiconv.a(libiconv.so.2) /gpfs/home/neteler/software/grass-7.0.svn/dist.powerpc-ibm-aix7.1.0.0/lib/libgrass_datetime.7.0.svn.so /usr/lib/libz.a(libz.so.1) /unix /usr/lib/libcrypt.a(shr_64.o) Cannot find libstdc++.a(libstdc++.so.6) /gpfs/home/neteler/software/grass-7.0.svn/dist.powerpc-ibm-aix7.1.0.0/lib/libgrass_dbmiclient.7.0.svn.so /gpfs/home/neteler/software/grass-7.0.svn/dist.powerpc-ibm-aix7.1.0.0/lib/libgrass_dbmibase.7.0.svn.so /gpfs/home/neteler/software/grass-7.0.svn/dist.powerpc-ibm-aix7.1.0.0/lib/libgrass_dgl.7.0.svn.so /gpfs/home/neteler/software/grass-7.0.svn/dist.powerpc-ibm-aix7.1.0.0/lib/libgrass_dig2.7.0.svn.so /gpfs/home/neteler/software/grass-7.0.svn/dist.powerpc-ibm-aix7.1.0.0/lib/libgrass_linkm.7.0.svn.so /gpfs/home/neteler/software/grass-7.0.svn/dist.powerpc-ibm-aix7.1.0.0/lib/libgrass_rtree.7.0.svn.so /gpfs/home/neteler/software/grass-7.0.svn/dist.powerpc-ibm-aix7.1.0.0/lib/libgrass_btree2.7.0.svn.so /gpfs/home/neteler/software/grass-7.0.svn/dist.powerpc-ibm-aix7.1.0.0/lib/libgrass_gproj.7.0.svn.so Cannot find libgcc_s.a(shr.o) /usr/lib/libc.a(shr.o) Cannot find libstdc++.a(libstdc++.so.6) Cannot find libgcc_s.a(shr.o) /usr/lib/libcrypt.a(shr.o) libgrass_gproj.7.0.svn.so needs: Cannot find libstdc++.a(libstdc++.so.6) /usr/lib/libpthread.a(shr_xpg5_64.o) /opt/freeware/lib/libiconv.a(libiconv.so.2) /gpfs/home/neteler/software/grass-7.0.svn/dist.powerpc-ibm-aix7.1.0.0/lib/libgrass_gis.7.0.svn.so Cannot find libgcc_s.a(shr.o) /usr/lib/libc.a(shr_64.o) /unix /usr/lib/libc.a(shr.o) /gpfs/home/neteler/software/grass-7.0.svn/dist.powerpc-ibm-aix7.1.0.0/lib/libgrass_datetime.7.0.svn.so /usr/lib/libz.a(libz.so.1) /usr/lib/libcrypt.a(shr_64.o) /usr/lib/libcrypt.a(shr.o) The complete list I have put here: http://pastebin.com/VXv9nGkG So two more libs need to somehow be added. Maybe a simple Makefile magic but I don't see what to do. Markus ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] compilation of grass on AIX 7.1
On Mon, Nov 25, 2013 at 10:25 AM, Markus Metz markus.metz.gisw...@gmail.com wrote: if some libraries are in a non-standard location, you might need to add -Wl,-bsvr4,-R,/opt/freeware/lib or equivalent e.g. -Wl,-R,/opt/freeware/lib/gcc/powerpc-ibm-aix6.1.0.0/4.2.0/ppc64/ to LDFLAGS [0]. Markus M [0] http://grasswiki.osgeo.org/wiki/Compile_and_Install#AIX On Sun, Nov 24, 2013 at 8:16 PM, Markus Neteler nete...@osgeo.org wrote: On Sun, Nov 24, 2013 at 5:07 PM, Glynn Clements gl...@gclements.plus.com wrote: Markus Neteler wrote: Any hints how to inject libstdc++.so for those modules which depend on $(GPROJLIB)? Maybe a trivial issue but I just don't know what to change. AFAICT, libgrass_gproj only requires libstdc++ because of GDAL. Yes, I see. So the definition of GDALLIBS in Platform.make should include -lstdc++ (and anything else which is required for C++ support). Unfortunately adding it: svn diff include/ Index: include/Make/Platform.make.in === --- include/Make/Platform.make.in (revision 58291) +++ include/Make/Platform.make.in (working copy) @@ -157,7 +157,7 @@ LAPACKINC = @LAPACKINC@ #GDAL/OGR -GDALLIBS= @GDAL_LIBS@ +GDALLIBS= @GDAL_LIBS@ -lstdc++ GDALCFLAGS = @GDAL_CFLAGS@ USE_GDAL= @USE_GDAL@ USE_OGR = @USE_OGR@ ... does not help. I checked, it is present in include/Make/Platform.make However, I don't understand why this would only affect libgrass_gproj and not other libraries which require GDAL. Indeed it does, I looped over it now. Here the reduced list of failure (the others ldd checks report no error): libgrass_cluster.7.0.svn.so needs: /gpfs/home/neteler/software/grass-7.0.svn/dist.powerpc-ibm-aix7.1.0.0/lib/libgrass_imagery.7.0.svn.so /gpfs/home/neteler/software/grass-7.0.svn/dist.powerpc-ibm-aix7.1.0.0/lib/libgrass_raster.7.0.svn.so /gpfs/home/neteler/software/grass-7.0.svn/dist.powerpc-ibm-aix7.1.0.0/lib/libgrass_gis.7.0.svn.so /usr/lib/libc.a(shr_64.o) /usr/lib/libpthread.a(shr_xpg5_64.o) /gpfs/home/neteler/software/grass-7.0.svn/dist.powerpc-ibm-aix7.1.0.0/lib/libgrass_vector.7.0.svn.so /opt/freeware/lib/libiconv.a(libiconv.so.2) /gpfs/home/neteler/software/grass-7.0.svn/dist.powerpc-ibm-aix7.1.0.0/lib/libgrass_datetime.7.0.svn.so /usr/lib/libz.a(libz.so.1) /unix /usr/lib/libcrypt.a(shr_64.o) Cannot find libstdc++.a(libstdc++.so.6) /gpfs/home/neteler/software/grass-7.0.svn/dist.powerpc-ibm-aix7.1.0.0/lib/libgrass_dbmiclient.7.0.svn.so /gpfs/home/neteler/software/grass-7.0.svn/dist.powerpc-ibm-aix7.1.0.0/lib/libgrass_dbmibase.7.0.svn.so /gpfs/home/neteler/software/grass-7.0.svn/dist.powerpc-ibm-aix7.1.0.0/lib/libgrass_dgl.7.0.svn.so /gpfs/home/neteler/software/grass-7.0.svn/dist.powerpc-ibm-aix7.1.0.0/lib/libgrass_dig2.7.0.svn.so /gpfs/home/neteler/software/grass-7.0.svn/dist.powerpc-ibm-aix7.1.0.0/lib/libgrass_linkm.7.0.svn.so /gpfs/home/neteler/software/grass-7.0.svn/dist.powerpc-ibm-aix7.1.0.0/lib/libgrass_rtree.7.0.svn.so /gpfs/home/neteler/software/grass-7.0.svn/dist.powerpc-ibm-aix7.1.0.0/lib/libgrass_btree2.7.0.svn.so /gpfs/home/neteler/software/grass-7.0.svn/dist.powerpc-ibm-aix7.1.0.0/lib/libgrass_gproj.7.0.svn.so Cannot find libgcc_s.a(shr.o) /usr/lib/libc.a(shr.o) Cannot find libstdc++.a(libstdc++.so.6) Cannot find libgcc_s.a(shr.o) /usr/lib/libcrypt.a(shr.o) libgrass_gproj.7.0.svn.so needs: Cannot find libstdc++.a(libstdc++.so.6) /usr/lib/libpthread.a(shr_xpg5_64.o) /opt/freeware/lib/libiconv.a(libiconv.so.2) /gpfs/home/neteler/software/grass-7.0.svn/dist.powerpc-ibm-aix7.1.0.0/lib/libgrass_gis.7.0.svn.so Cannot find libgcc_s.a(shr.o) /usr/lib/libc.a(shr_64.o) /unix /usr/lib/libc.a(shr.o) /gpfs/home/neteler/software/grass-7.0.svn/dist.powerpc-ibm-aix7.1.0.0/lib/libgrass_datetime.7.0.svn.so /usr/lib/libz.a(libz.so.1) /usr/lib/libcrypt.a(shr_64.o) /usr/lib/libcrypt.a(shr.o) The complete list I have put here: http://pastebin.com/VXv9nGkG So two more libs need to somehow be added. Maybe a simple Makefile magic but I don't see what to do. Markus ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] Exporting maps as multilayer GeoTIFF - layer limit?
Markus Neteler wrote: Hi, I struggle to export a time series as multilayer GeoTIFF: it stops at layer 145. I have generated a time series with 365 maps and added all into a group. Exporting that in order to export as multilayer GeoTIFF, it fails: GRASS 7.0.svn (nc_spm_08):~ r.out.gdal ... ... 100% 100% 100% Exporting raster data to GTiff format... ERROR 6: SetColorTable() not supported for multi-sample TIFF files. 100% ... ERROR: Unable to open input file /home/neteler/grassdata/nc_spm_08/user1/cell_misc/testmap.145/f_format You were bitten by the ulimit of your system (ulimit -n). Actually it was a bug in r.out.gdal which did not close raster maps after export. Fixed in r58304. Markus M Where is the trick? I have added a small script to generate a synthetic time series for debugging. If it matters: GRASS 7.0.svn (nc_spm_08):~ ulimit unlimited It happens in a different location as well with my original dataset. Hints welcome, Markus ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] i.pca doesn't have the '--o' flag
On Thu, Nov 21, 2013 at 11:09 PM, Markus Neteler nete...@osgeo.org wrote: On Thu, Nov 21, 2013 at 10:48 PM, Nikos Alexandris n...@nikosalexandris.net wrote: Shouldn't i.pca have, in both G64x and G7, the overwrite flag? Ideally yes. It will be lacking due to the prefix= approach used therein. The same applies to r.texture. Fixed in r58311. Markus M On the contrary: r.lake offers two overwrite flags :p Markus ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] Aborted (core dumped), during v.build
It looks very much like an out-of-memory error. In the gdb backtrace I see #4 0x76601ebd in RTreeNewListBranch (t=0x609c30) at node.c:441 p = 0x0 which means that memory allocation failed. Try export GRASS_VECTOR_LOWMEM=1 r.to.vect --o --v input=seg_0.05_final@pietro output=seg005 type=area Markus M ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] v.clean tool=break fails
On Sat, Nov 30, 2013 at 12:26 PM, Martin Landa landa.mar...@gmail.com wrote: Hi all, recently I found problem when breaking lines using `v.clean`. I have a patched map that contains boundaries and centroids. $ v.info x3 -t nodes=8219 points=0 lines=0 boundaries=11807 centroids=4876 areas=4875 islands=1222 primitives=16683 map3d=0 When I try to break lines on intersections `v.clean` fails because it tries to read nodes from centroids (which obviously fails). $ v.clean in=x3 out=x4 tool=break --o ... Tool: Break lines at intersections 0..ERROR: Nodes not available for line 11725 Defining `type` explicitly helps $ v.clean in=x3 out=x4 tool=break --o type=boundary Fixed in r58340, please test. Markus M ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] v.clean tool=break fails
On Sat, Nov 30, 2013 at 6:02 PM, Martin Landa landa.mar...@gmail.com wrote: Hi Markus, 2013/11/30 Markus Metz markus.metz.gisw...@gmail.com: [...] Tool: Break lines at intersections 0..ERROR: Nodes not available for line 11725 Defining `type` explicitly helps $ v.clean in=x3 out=x4 tool=break --o type=boundary Fixed in r58340, please test. thanks for super-quick fix! It seems to be OK, I found similar problem in `merge_lines.c` (see the attached patch). How did you discover that bug in Vect_merge_lines()? To my knowledge all modules calling Vect_merge_lines() set the correct type (lines and/or boundaries). Markus M ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] i.segment: Invalid region id -1
On Tue, Dec 3, 2013 at 1:05 AM, Nikos Alexandris n...@nikosalexandris.net wrote: Nikos Alexandris wrote: .. I've tested this with at least three different similar cases. All work fine without the seed map! All fail with a seed map supplied. I guess, the only real difference is time for the processes to complete, right? OK, I've just discovered that I mixed 8-bit (the Pan images) for the seed map and 16-bit (the Multi-Spectral images). So, seed - 8-bit, group to be segmented - 16-bit. Does this play a role? No, the seed map must be integer (not more than 32 bit int), that's the only limitation. The data to be segmented can be anything, integer and/or floating point. But it does not make sense to use a pan band as seeds when segmenting the other bands. Seeds are typically the result of a previous run of i.segment or the result of a previous classification of the same data. Markus M ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] v.clean tool=break fails
On Mon, Dec 2, 2013 at 4:21 AM, Martin Landa landa.mar...@gmail.com wrote: Hi Markus, 2013/11/30 Markus Metz markus.metz.gisw...@gmail.com: How did you discover that bug in Vect_merge_lines()? To my knowledge all modules calling Vect_merge_lines() set the correct type (lines and/or boundaries). by `v.clean tool=snap -c`, see [1]. Oops. Thanks for spotting this! Markus M Martin [1] http://trac.osgeo.org/grass/browser/grass/trunk/vector/v.clean/main.c#L407 ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] i.segment: Invalid region id -1
Nikos Alexandris wrote: Nikos Alexandris wrote: I've tested this with at least three different similar cases. All work fine without the seed map! All fail with a seed map supplied. I guess, the only real difference is time for the processes to complete, right? OK, I've just discovered that I mixed 8-bit (the Pan images) for the seed map and 16-bit (the Multi-Spectral images). So, seed - 8-bit, group to be segmented - 16-bit. Does this play a role? Markus Metz: No, the seed map must be integer (not more than 32 bit int), that's the only limitation. The data to be segmented can be anything, integer and/or floating point. Good to know. I guess this is because the segments are only counted then. Not exactly. You grow stuff from seeds, in this case segments. A seeds map defines initial segments (objects) which are then grown if possible. Internally, a seeds map is processed just as in r.clump: all contiguous cells with the same id will belong to the same initial segment. But it does not make sense to use a pan band as seeds when segmenting the other bands. Seeds are typically the result of a previous run of i.segment or the result of a previous classification of the same data. Then I have inserted a small mistake in my tests/workflow. Wanted to drive finer objects (from Pan) in bigger ones (based on MS). Will adjust. The seeds map does the opposite. You probably want to do pansharpening first. Anyway, it does not explain why the invalid region id -1 has been encountered, which should not happen. Markus M ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] i.segment: Invalid region id -1
Nikos Alexandris wrote: Markus M: But it does not make sense to use a pan band as seeds when segmenting the other bands. Seeds are typically the result of a previous run of i.segment or the result of a previous classification of the same data. Nikos A: Then I have inserted a small mistake in my tests/workflow. Wanted to drive finer objects (from Pan) in bigger ones (based on MS). Will adjust. MM: The seeds map does the opposite. You probably want to do pansharpening first. NA: Ha-Yey :D I did (in some cases). Just for completeness, ehm... I was too fast. So, I did use sharpenned images only in 2 trials, however, as I can actually see in the history. The exact same process in two different Mapsets (same Location), QuickBird2 data: --%-- i.segment msx_hpf out=segments_msx_hpf_seeded_t0.02 threshold=0.02 minsize=4 seed=segments_pan_t0.01 memory=3000 iterations=1000 --%-- It worked in one case (repeated to be sure) and it failed in another! Different computational regions? In the failing case, before the ERROR message, there are multiple WARNINGS issued: .. WARNING: Region consists of only one cell, nothing to update This should not happen, a bug in the region growing algorithm. Now, I have re-ran the failed one and I get this strange: .. 0..5..10..15..20..25..30..35..40..ERROR: Invalid region id -1489 Essentially the same like ERROR: Invalid region id -1. Again, this should not happen. .. I went after looking all of the details of the involved maps. The only strange thing I can see (which I caused) is that the region is 0.6, the seed (segments_pan_t0.01) is also 0.6 while the group of Pan-Sharpened images are (each) of 0.60017817 (ns) x 0.60016801 (we) resolution. Is this my mistake? The resolution(s) should be identical, right? Yes. I guess in the process of pansharpening, the region was set to 0.6, then the resolution was adjusted to the extents (for g.region, extents have precedence over resolution). The correct way of adjusting the region would be to either set the region to the pan band or align the region to the band band (g.region align=pan). Note that g.region res=0.6 -a can introduce a pixel shift. Can you reproduce that with sample data? Or give me chance to reproduce that? Markus M ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] PCA (i.pca) in G7: filtering and rescaling
On Wed, Dec 4, 2013 at 3:42 PM, Nikos Alexandris n...@nikosalexandris.net wrote: Nikos Alexandris wrote: 1) I can't see any differences in the derived Principal Components .. okay, to clarify: I mean the resulting images which, initially are Principal Components (synthetic images) and, after applying filtering backward PCA, the resulting images approach the original data -- still they are modified. Example (using i.pca in G7, though r.info below executed from G64): .. PC1 2.78 ( 0.4947, 0.5922, 0.5743, 0.2735) [69.53%] .. Ignore please. I picked the wrong example. It works with percentage = 99 for example. Because 70 + 20 = 90 was still inside the 1st component (=96.52). Sorry for the noise. Then I need to see why the rescaling is not getting at [0,255]. Rescaling applies to PC scores. With filtering, the output is not PC scores, but a filtered version of the input. If you want the filtered output to be rescaled, please use r.recode afterwards. ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] PCA (i.pca) in G7: filtering and rescaling
On Thu, Dec 5, 2013 at 1:15 PM, Nikos Alexandris n...@nikosalexandris.net wrote: Nikos Alexandris: ...we need those extra digits to make it easy rejecting last Principal Component(s) prior to the backward PCA. Might be one, two or numerous (?) depending on the dimensions. Markus M: I think it rather depends in the amount of information encoded in each PC. It does. PCA works on global stats so one has to go through, then study visuals and numbers, then decide what to keep or how to treat further. In my very simple example, I want to see whether I want to reject the last or the two last ones. If the filtering option lets me do that, I am happy :-). To exemplify, currently I can't reject two last components whicih account for 0.06 and 0.21 of the original data variance. I tested yesterday and the filter does not differentiate those subtle details which might be of importance (for a subsequent classification of High-Res images). I don't think filtering makes sense in this case. If important information is encoded in components explaining only a small part of the variance, you don't want to filter them out. The idea of the filtering option is to discard components that do not contribute much in terms of observed variance. It sounds like you rather want to identify those PCs that encode information that is important in your case. In this case filtering does not make sense, instead the identified components could be used for subsequent processing. Alternatively, PC selection could also be based on the Eigenvalue, typically all PCs with an Eigenvalue = 1 (centered and scaled input) would be used. It depends. Typically might be simply compressing data or reducing salt 'n' pepper. However, in change detection studies, where changes are likely to appear in higher order components, it's not uncommon to have several components which account for = 1 of the original variance and still are the ones you really need. Don't filter in these cases, use the components relevant for the study ? You can also filter manually using the factor loadings of the components of interest. Markus M ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] G7: v.loutlier Decomposition failed
On Fri, Dec 6, 2013 at 1:17 PM, Blumentrath, Stefan stefan.blumentr...@nina.no wrote: Hi Maybe it is not of importance, because no one else would use such extreme settings, but in case someone considers it as a bug: I was testing a bit with v.outlier in GRASS 7 on dense LiDAR data (~4 points per m2). When I use a very low Tykhonov regularization parameter (lambda_i) of 0.001 and rather short spline steps (1.5) at 0.5m resolution I get an error-message: “Decomposition failed at row 16499 and col 0” I found similar errors with very small lambda values, i guess it's some floating point precision limit. It seems that lambda should not be much smaller than 0.0001. For v.outlier, I would leave lambda at the default and rather use shorter spline steps, if you want to fine-tune the results. Markus M ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] move v.polytoline?
On Tue, Dec 10, 2013 at 11:14 AM, Luca Delucchi lucadel...@gmail.com wrote: Hi devs, Some days ago I create a really simple script to convert polygon to line [0]. I would like to know what do you think to move it to main code? I'm not sure if it could be useful for end user or not let me know A suggestion for a small improvement: it would be nice if the information to which areas the lines belong to would be kept as attributes (v.to.db option=sides). Markus M Thanks [0] http://trac.osgeo.org/grass/browser/grass-addons/grass7/vector/v.polytoline -- ciao Luca http://gis.cri.fmach.it/delucchi/ www.lucadelu.org ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] v.clean problem with rmarea (ERROR: Area merging failed)
On Tue, Dec 31, 2013 at 5:23 AM, Yann Chemin yche...@gmail.com wrote: Hi, (SVN G7) it does work with thres inferior to 600, but not above... --- v.clean input=vnir4567_seg_0 output=vnir4567_seg_0_no_small_areas type=area tool=rmarea thres=1.00 --overwrite -- Tool: Threshold Remove small areas: 1 -- WARNING: Vector map vnir4567_seg_0_no_small_areas already exists and will be overwritten Copying features... Rebuilding parts of topology... Building topology for vector map vnir4567_seg_0_no_small_areas@PERMANENT... Registering primitives... 19070 primitives registered 187624 vertices registered Building areas... 5344 areas built 251 isles built Attaching islands... Attaching centroids... Number of nodes: 8636 Number of primitives: 19070 Number of points: 0 Number of lines: 0 Number of boundaries: 13729 Number of centroids: 5341 Number of areas: 5344 Number of isles: 251 -- Tool: Remove small areas ERROR: Area merging failed Should be fixed in r58647. Markus M ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] [GRASS-user] GCP GUI error
On Fri, Jan 10, 2014 at 1:31 AM, Ahmadou Dicko dicko.ahma...@gmail.com wrote: Hi, I'm using the GRASS GIS 7 : g.version -g version=7.0.svn date=2014 revision=58660 build_date=2014-01-10 And since my last build (today) I can't use properly the gcp gui, more precisely I can't select a maps. When I launch the gcp gui (from the console) and try to select a mapset, I have this error message : File /usr/local/grass-7.0.svn/etc/gui/wxpython/gcp/manager.py, line 390, in OnLocation self.cb_mapset.SetItems(self.mapsetList) AttributeError: 'MapsetSelect' object has no attribute 'SetItems' I don't know if it's a bug or a problem with my setup and I hope that I gave enough information to help to eventually fix this. It seems that this is more general bug in the GUI, because a similar bug appears with r.proj and v.proj, probably caused by recent changes to gui/wxpython/gui_core/gselect.py Markus M Thanks -- Ahmadou H. DICKO statistician economist (Ingénieur Statisticien Économiste) PhD candidate in Climate change economics Faculty of economics and managment - Cheikh Anta Diop University West African Science Service Center on Climate Change and Adaptated Land Use (WASCAL) Center for Development Research (ZEF) - University of Bonn email : ahmadou.di...@ucad.edu.sn twitter : @dickoah github : github/dickoa tel : +221 33 827 55 16 portable: +221 77 123 81 69 ___ grass-user mailing list grass-u...@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-user ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] [GRASS-user] GCP GUI error
On Sat, Jan 11, 2014 at 6:44 PM, Martin Landa landa.mar...@gmail.com wrote: Hi, 2014/1/11 Ahmadou Dicko dicko.ahma...@gmail.com: Not the first page but at the second page when you click on 'create or edit group'. I just have two mapsets in my source location (one + PERMANENT). right, should be fixed in r58675. I will check other parts of the wxGUI for similar problem. Martin The GUI for [r|v].ptoj is still not working: the list of mapsets does not match the selected location. Markus M ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] [GRASS-user] GCP GUI error
On Sun, Jan 12, 2014 at 12:10 PM, Martin Landa landa.mar...@gmail.com wrote: Hi, 2014/1/12 Markus Metz markus.metz.gisw...@gmail.com: The GUI for [r|v].ptoj is still not working: the list of mapsets does not match the selected location. thanks for reminder, should be fixed r58679. Anyway this part of forms.py needs to be complete rewriten using signals. The GUI for [r|v].proj is still not working: the list of mapsets is now correct, but the list of raster maps is wrong. Please test before committing changes. Markus M ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] weren't r.stream going to core? was: r.stream.* problems on Ubuntu when installed via g.extensions
On Sun, Jan 19, 2014 at 9:08 PM, Helena Mitasova hmit...@ncsu.edu wrote: On Jan 19, 2014, at 3:01 PM, Martin Landa wrote: Hi, 2014/1/19 Helena Mitasova hmit...@ncsu.edu: Maybe it just waits for somebody to put it there, we would like to see it in the core as well. r.stream.extract is already in core, the other modules are scheduled to follow. it's not only question of putting it here. The modules need some review (code, parameters, manuals). I will be able to help with this procedure in some days. There will be hopefully more people interested. Martin If that is the case, that is a completely different issue, I thought they are ready. I agree with Martin that the modules need some review. E.g. I encountered a bug in r.stream.order where some stream segments got order 0 (zero) which is invalid. I am also interested in having these modules in core and want to help with the review as soon as I find some time... Markus M We can look at how much work it would be. Then what about v.transects - it has been updated to grass7 and tested in class and has a manual and testing report available. Can this module go to core? How do we make the decision for ready to go modules? Helena ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] Handling of Python scripts on MS Windows
Helmut Kudrnovsky wrote: Moritz Lennert wrote Glynn Clements: Where it gets problematic is if the user already has a Python installation but it's not suitable for whatever reason. In the worst case they may be faced with a choice between using GRASS or using whatever the existing Python was installed for. IMHO keeping in mind that many GIS-interested winGRASS-users may already have been installed other (GIS-)software including a system-wide python installation, that will be the demanding challenge to provide a suitable solution ... Therefore I would suggest to keep using the embedded Python version which is known to work and includes all required components. Wingrass would need *.bat wrapper scripts as for GRASS6 in order to make sure that GRASS7 Python scripts are called with GRASS_PYTHON. One reason for using Python is that many users are familiar with Python and can easily write their own scripts. Writing Python scripts is made easier if a Wingrass user associates Python scripts not with Python but with the text editor of choice, in which case the system handler will not execute Python scripts, but edit them. Now the task is to not only make writing Python scripts easy, but also executing them. This is done via File - Launch script. There is no reason to assume that any system-wide .py association is suitable to execute GRASS Python scripts, thus the easiest way out might be to just ignore it. my 2c Markus M ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] Handling of Python scripts on MS Windows
On Sat, Jan 25, 2014 at 3:59 PM, Helena Mitasova hmit...@ncsu.edu wrote: Just a note, given that most of these problems were caused by conflicts with python installed by ArcGIS, I checked with our GIS support and the latest ArcGIS 10.2 installs python2.7 which (I guess) should work with GRASS. No, there are different versions of Python 2.7, and not all work with GRASS, see e.g. ticket 2015 On Jan 24, 2014, at 3:03 PM, Glynn Clements wrote: Markus Metz wrote: Where it gets problematic is if the user already has a Python installation but it's not suitable for whatever reason. In the worst case they may be faced with a choice between using GRASS or using whatever the existing Python was installed for. IMHO keeping in mind that many GIS-interested winGRASS-users may already have been installed other (GIS-)software including a system-wide python installation, that will be the demanding challenge to provide a suitable solution ... How many users will have versions which are: a) too old for GRASS to use, and b) required to be that old by the existing package? Bear in mind that GRASS won't be the only package affected by an outdated system-wide Python installation. GRASS should not use a system-wide Python installation, or more precisely, should explicitly ignore any system-wide python file associations. Expecting system-wide python file associations is the cause of all the trouble. The embedded but not installed Python version coming GRASS works fine and should IMHO be used for scripts via @%GRASS_PYTHON% %GISBASE%/scripts/SCRIPT_NAME %* AFAIK, the primary case where another package requires a system-wide Python installation is ArcGIS, no? IIRC, ArcGIS requires Python 2.6; is there some fundamental reason why this isn't suitable for GRASS? If so, does ArcGIS require that exact version, or can it use a later version? We can not care for all cases of other software (versions) relying on a system-wide python installation. yes, given that most of these problems were caused by conflicts with python installed by ArcGIS, I checked with our GIS support and the latest ArcGIS 10.2 installs python2.7 which should work with GRASS (correct me if I am wrong). We had similar situation with ArcGIS9* and GRASS6.3 where you had to use a specific winpython2.5 build 212 to have both of them working on the same machine. The test suggested by Markus in the related message would be still useful, but upgrading to ArcGIS10.2 should solve the problem? You can hardly suggest users to upgrade ArcGIS if they want to use GRASS... Therefore I would suggest to keep using the embedded Python version which is known to work Actually, it known not to work, hence this thread. It works as long as system-wide python file associations are ignored. Other Python versions might not work. The only way you can make execution of Python scripts seamless (i.e. works with system(), subprocess.Popen(), etc) is for the .py extension to be associated with a suitable interpreter (or launcher) in the registry. I disagree. For example, shell scripts work just fine in GRASS 6.4, even though there is no associated suitable interpreter (or launcher) in the registry. Any other approach will trap us into an endless maintenance burden, identifying cases where we have to provide special handling for Python scripts then implementing that handling. We did so for shell scripts, AFAIK this works. Markus M ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] Handling of Python scripts on MS Windows
On Mon, Jan 27, 2014 at 12:16 AM, Glynn Clements gl...@gclements.plus.com wrote: Markus Metz wrote: On Sat, Jan 25, 2014 at 3:59 PM, Helena Mitasova hmit...@ncsu.edu wrote: Just a note, given that most of these problems were caused by conflicts with python installed by ArcGIS, I checked with our GIS support and the latest ArcGIS 10.2 installs python2.7 which (I guess) should work with GRASS. No, there are different versions of Python 2.7, and not all work with GRASS, see e.g. ticket 2015 Any version of Python 2.7 should be suitable for GRASS. Not all versions of Python 2.7 are suitable for WinGRASS, see ticket #2150. However, it needs to be installed correctly, and GRASS needs to not attempt to bundle Python. GRASS should not use a system-wide Python installation, or more precisely, should explicitly ignore any system-wide python file associations. Expecting system-wide python file associations is the cause of all the trouble. Executing a script uses the registry associations for the script's extension. WinGRASS does not set registry associations for Python scripts, nor does it install Python system-wide. This is because we do not want to modify an existing Python installation. Attempting to by-pass the system's script execution mechanism (by explicitly executing Python scripts via a specific interpreter) is the cause of all the trouble. I disagree. Troubles arise if the system's interpreter, e.g. Python installed by ArcGIS, is used instead of the python version embedded in WinGRASS. Markus M ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] Handling of Python scripts on MS Windows
On Tue, Jan 28, 2014 at 5:13 PM, Pietro peter.z...@gmail.com wrote: On Tue, Jan 28, 2014 at 3:52 PM, Moritz Lennert mlenn...@club.worldonline.be wrote: On 28/01/14 16:07, Vaclav Petras wrote: The current problem is that there is a incompatibility between Python 2.7.3 and 2.7.4; some import fails because older system Python library (dll) does not contain MAXREPEAT variable. (To be honest, now I'm not sure how (dll) library is involved in this. I remember that even our python.exe was using dll from system Python...) I think this is the main issue: AFAIU, unless we completely isolate the GRASS Python environment from the outside, mixes such as can potentially happen. virtualenv might be a solution to this, but I don't know it enough. I'm using virtualenv daily to isolate GRASS on my system (linux), but still I'm not sure if could be the solution for windows users. I guess could be a solution if we create a setup.py file using setuptools [0] transforming GRASS into a normal python package (like: numpy, gdal, proj4, etc.). In this way all the Mac/Linux/windows users should be able to install GRASS and all its dependencies with: $ easy_install grass or using pip $ pip install grass I am afraid this does not work if there is no system-wide python installation. Moreover, how would easy_install and pip know that they should use GRASS_PYTHON? I mean, $ pip install grass could then only work if GRASS with GRASS_PYTHON is already installed? Markus M But I've never used setuptools so I don't have the knowledge to set and configure a setup.py file for such complex project like GRASS. Pietro [0] https://pypi.python.org/pypi/setuptools ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] Handling of Python scripts on MS Windows
Trying a summary on this discussion. AFAIU, the whole discussion boils down to the question if we want to require a system-wide installation of Python with correct python file associations in the registry, potentially deactivating an existing Python installation, or not. There seems to be agreement that we do not want to modify any existing system-wide python installation. That means that WinGRASS should 1) not do a system-wide installation of Python 2) not require a system-wide Python installation 3) explicitly ignore any existing system-wide installation of Python, or any Python file associations in the registry (text-editor, whatever) That means that we must assume that Windows has no idea what to do with Python files, and if it has an idea, this is most probably a bad idea, as far as GRASS scripts are concerned. As for shell scripts in GRASS 6. That in turn means that GRASS Python scripts must be called explicitly with GRASS_PYTHON, most importantly scripts from within scripts. IMHO, creating .bat files for python scripts is easy because the equivalent is already working for WinGRASS 6.4 where .bat files are created for shell scripts. AFAICT none of the other suggestions (virtualenv, Python launcher, pip, etc) has been tested successfully on Windows with GRASS 7, both with and without a separate incompatible system-wide Python installation. Markus M ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
