[GRASS-user] v.to.rast to get nice smooth vector-based pngs ?
Hi, Thanks to the support of this mailing list, I managed to paste and import a huge amount of NASA swbd tiles into a GRASS vector-layer, aimed at completing the hydrography of my relief-map. My initial plan was to export the GRASS vector layer to svg and rework it with inkscape, before final rasterization, and merging with the topographic layer. Unfortunately, the exported svg map is far to big to be edited with inkscape, at least on my hardware (AMD64, 3Gb ram and about 350Gb hdd). My first attempt was to simplify the vector layer first, using v.simplify, but it produces weired results by mixing water and land areas. A possible explanation for this would be the -c flag I used during the import operation, which skipped the usual cleaning process, mainly because that cleaning already produced weired results. Second choice would be a GRASS-lead direct rasterization; I mean, given the tools offered by this GIS, why using inkscape. After all, all I want is a nice anti-aliased raster, based on the vector layer, with blue lines (rivers, coastlines), and light-blue areas (water bodies). I learned about v.to.rast again, thanks to this mailing list, but unfortunately, I don't really know how to use it. All my testings result in a white raster, showing only green waterbodies. I tried to investigate this, and found that v.to.rast seems to rely heavily on the attribute table of my vector map - and it looks like it (attribute table) contains ONLY areas with centroids, eg waterbodies: there are only two colums, first one cat number (1-52492), and second one entitled FACC_CODE, with either value 'BA040' or 'BH080' ___ grass-user mailing list grass-user@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-user
[GRASS-user] v.to.rast to get nice smooth vector-based pngs ?
NOTE: I'm really sorry for the double-post, but the previous mail was uncomplete. Could somebody delete it ? Hi, Thanks to the support of this mailing list, I managed to paste and import a huge amount of NASA swbd tiles into a single GRASS vector-layer, aimed at completing the hydrography (rivers and coastlines) of my relief-map. My initial plan was to export the GRASS vector layer to svg and rework it with inkscape, before final rasterization, and merging with the topographic layer. Unfortunately, the exported svg file is far to big to be edited with inkscape, at least on my hardware (AMD64, 3Gb ram and about 350Gb hdd). My first attempt was to simplify the vector layer first, using v.simplify, but it produces weired results by mixing water and land areas. A possible explanation for this would be the -c flag I used during the import operation, which skipped the usual cleaning process, mainly because that cleaning already produced weired results. Second choice would be a GRASS-lead direct rasterization; I mean, given the tools offered by this GIS, why using inkscape ? After all, all I want is a nice raster, based on the vector layer, with blue lines (rivers, coastlines), and light-blue areas (water bodies): nothing more than an anti-aliased d.vect display. I learned about v.to.rast, again thanks to this mailing list, but unfortunately, I don't really know how to use it. All my testings result in a white raster, showing only green waterbodies. I tried to investigate this, and found that v.to.rast seems to rely heavily on the attribute table of my vector map - and it looks like it (the attribute table) contains ONLY areas with centroids, eg waterbodies: there are only two colums, first one with cat numbers (1-52492), and second one entitled FACC_CODE, with either value 'BA040', 'BH140' or 'BH080'. I have no clue about what these numbers mean, as all categories, once highlined, seem to refer to centroids (I may be wrong, because I did'nt test each one of the 53k). Thus my question is quite simple: is it possible to rasterize my vector in GRASS, using the same colors as in d.vect -color and -fcolor options, and than to export it to png ? How would you do this ? Would the results require more editing, or ready-for-merging with the topographic layer ? Any help here would be great; perhaps just a link to a good tutorial about this issue. Obvioulsy, third choice would be to cut the GRASS vector layer in small piece, export each one to svg an rasterize them with inkscape, but I guess this would take me a huge amount of time... Also I'm not sure about the modus operandi here. Thank you very much for your patience and your help, Felix ___ grass-user mailing list grass-user@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-user
[GRASS-user] Stranges things after importing a shapefile
NOTICE: This is a second mail, sent because I didn't pay attention to the size limit of this mailing list. I replaced the attached image by imagehack links. Hi all, While still working on my huge map of Europe, I've noticed many differences between the original shapefile and the grass vector layer created after import. To visualize the problem, I took two Screenshots: -First one is from gqis 1.01, showing the raw shapefile: http://img214.imageshack.us/img214/4884/qgis.png -Second one from grass 6.4RC5, showing the imported vector layer: http://img4.imageshack.us/img4/8462/grassgis.png Notice how many areas seem broken in the grass layer, and how entire parts of large rivers (take the rhine, for an example) are missing. The initial shapefile was patched together (from a few hundred nasa swbd tiles) using Markus Neteler's script (Thanks again Markus !), and seem to produce good results, at least in Qgis. The import command I used was: v.in.ogr -z -o dsn=/DATA/swbd/shp/tiles/coastlines.shp output=coastlines I somehow had to override the projection, because grass doesn't seem to recognize the projection of the shapefile - although both are of the same WSG84 LAT/LON proj. My question is: what went wrong ? Perhaps there is an import step I'm missing, since once the 'clean' command finishes, I get an awful lot of areas without a centroid; I don't know. But it would be nice, for sure, if grass could just translate the shapefile, as it is, and allow me to export a *complete* vector map to inkscape. Thanks for your help, Felix ___ grass-user mailing list grass-user@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-user
Re: [GRASS-user] Stranges things after importing a shapefile
Ok: you are right, the pictures I uploaded are not very useful; so I made another 4 screenshots, to show my problem: First Image: Rhine river, and swiss lakes in qgis (displaying the raw shapefile): http://img690.imageshack.us/img690/1144/swbdrhineqgis.png Second Image: (Approximately) Same region, after importation as GRASS vector, using v.in.ogr: http://img690.imageshack.us/img690/3850/swbdrhinegrass.png The middle-rhine just vanished over the process ! Also, notice that no lake is recognized as closed area (although zooming in shows no apparent dangles), and doesn't color. This is annoying, since I would mean a manual rework all lakes to show them properly in the final map. Third image: Zoom at the middl-Rhine, with Qgis: http://img442.imageshack.us/img442/1650/swbdrhineqgis2.png Forth image: (Approximately) Same region, after importation as GRASS vector: http://img266.imageshack.us/img266/6029/swbdrhinegrass2.png Obvious question: what happened to the river ? As explined, the shapefile are meant to serve as vector layer on my final topographic map; thus I'm using grass as middle-step, between qgis shapefile and inkscape. Im very interested in nikos' import-without-clean option: would this work out for me (eg: would it be possible to export the non-cleaned grass vector layer to svg) ? Thanks for your help, Felix ___ grass-user mailing list grass-user@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-user
Re: [GRASS-user] Stranges things after importing a shapefile
It Works ! I used v.in.ogr -c command and now it looks pretty good: http://img25.imageshack.us/img25/6802/swbdgrass3.png Thanks you very much, Nikos. Now I got another problem: export to inkscape produce a way-to-big file, which I can't rework with my current system config (AMD64, 3Gb ram). Is it possible to do the job in grass, and produce a good-looking anti-aliased raster river layer, which I could simply layer over the topographic layer ? Is this doable with v.to.rast ? Thanks again for your help, Felix 2009/11/10 Νίκος Αλεξανδρής nikos.alexand...@felis.uni-freiburg.de: On Tue, 2009-11-10 at 17:50 +0100, Felix Schalck wrote: Ok: you are right, the pictures I uploaded are not very useful; so I made another 4 screenshots, to show my problem: First Image: Rhine river, and swiss lakes in qgis (displaying the raw shapefile): http://img690.imageshack.us/img690/1144/swbdrhineqgis.png Second Image: (Approximately) Same region, after importation as GRASS vector, using v.in.ogr: http://img690.imageshack.us/img690/3850/swbdrhinegrass.png The middle-rhine just vanished over the process ! Also, notice that no lake is recognized as closed area (although zooming in shows no apparent dangles), and doesn't color. This is annoying, since I would mean a manual rework all lakes to show them properly in the final map. Third image: Zoom at the middl-Rhine, with Qgis: http://img442.imageshack.us/img442/1650/swbdrhineqgis2.png Forth image: (Approximately) Same region, after importation as GRASS vector: http://img266.imageshack.us/img266/6029/swbdrhinegrass2.png Obvious question: what happened to the river ? Right, very clear. The problem is known. I might be missing something but, in general, I think my previous answer, and Achims suggestion, covers it. As explined, the shapefile are meant to serve as vector layer on my final topographic map; thus I'm using grass as middle-step, between qgis shapefile and inkscape. Im very interested in nikos' import-without-clean option: would this work out for me (eg: would it be possible to export the non-cleaned grass vector layer to svg) ? Yes (for a nice map but you won't be good for using this for analysis). Go ahead and try ;-) Nikos ___ grass-user mailing list grass-user@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-user
Re: [GRASS-user] Very high resolution topographic map of Europe: need help and advices: UPDATE
Looks like we all agree on this. I'm sure there was a good reason behind adopting ESRI file geodatabase, like the 2Gb file limit of previously used ms access dbs, but adopting ONLY ESRI formats is definitely not very public-friendly. Now, I'm sure that each new short message left int Mr Vogt Message box (juergen.vogt(-at/arobase-)jrc.ec.europa.eu), contributes to increase our chances to get the data published in a another format. Hermann Pfeffer, from the eea (european environment agency) just wrote me that an existing eu law obliging all eu-institutions to respond to public inquiries within 2 weeks, further increasing our chances to get an answer. Thank you all for your interest, Felix 2009/9/19 Nikos Alexandris nikos.alexand...@felis.uni-freiburg.de: On Sat, 2009-09-19 at 12:38 +, Benjamin Ducke wrote: It is entirely unacceptable that data produced with European public funding is available in one random, proprietary format only. Maybe we should all email them individually and ask for an open standard format, so they see that there is some wider interest in this. Ben +1 Nikos - Original Message - From: Felix Schalck felix.scha...@gmail.com To: Markus Neteler nete...@osgeo.org, benjamin ducke benjamin.du...@oxfordarch.co.uk Cc: grass-user grass-user@lists.osgeo.org Sent: Saturday, September 19, 2009 12:05:10 AM GMT +01:00 Amsterdam / Berlin / Bern / Rome / Stockholm / Vienna Subject: Re: [GRASS-user] Very high resolution topographic map of Europe: need help and advices: UPDATE Hi, I got news from the GDAL mailing list, where I posted a similar question about that strange format I downloaded on the CCM JRC web page. It seems to be the new ArcGIS file geodatabase format introduced by ESRI in ArcGIS 9.2, according to Jason Roberts and Frank Warmerdam. The important thing here is that it is a different database format (from .mdb or argis personal database), entirely proprietary which cannot, as of september 2009, be read by any OGR driver. So we took steps to contact the author/distributor of CCM data in order to have the set distributed in another format. More on this to follow. Regards, Felix ___ grass-user mailing list grass-user@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-user ___ grass-user mailing list grass-user@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-user
Re: [GRASS-user] Very high resolution topographic map of Europe: need help and advices: UPDATE
Hi, I got news from the GDAL mailing list, where I posted a similar question about that strange format I downloaded on the CCM JRC web page. It seems to be the new ArcGIS file geodatabase format introduced by ESRI in ArcGIS 9.2, according to Jason Roberts and Frank Warmerdam. The important thing here is that it is a different database format (from .mdb or argis personal database), entirely proprietary which cannot, as of september 2009, be read by any OGR driver. So we took steps to contact the author/distributor of CCM data in order to have the set distributed in another format. More on this to follow. Regards, Felix ___ grass-user mailing list grass-user@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-user
Re: [GRASS-user] Very high resolution topographic map of Europe: need help and advices: UPDATE
I frankly don't get it: the data I downloaded is definitely sort of a database, with each entry split among 4 differents files: entry.freelist,.gdbindexes,.gdbtable and .gdbtablx. ogrinfo deosn't seem to recognise any of the different files (but maybe I don't have the right driver compiled into (although PGEO is shown supported by ogrinfo -f). Anyway, I read somewhere about GRASS database support, and now I'm wondering if I don't have to import the whole fileset as a db first. Unfortunately, I don't have a clue on how to do this, so help would be greatly appreciated. Besides: I already followed your advice Markus, and wrote Mr. Vogt (whose mail is given on the CCM download page) and asked him if he would consider publishing the data in shapefile format (or anything else supported by GRASS). Thanks for your time and your patience, Felix 2009/9/14 Markus Neteler nete...@osgeo.org: On Mon, Sep 14, 2009 at 6:19 PM, Martin Landa landa.mar...@gmail.com wrote: 2009/9/14 Felix Schalck felix.scha...@gmail.com: [...] How do you import ArcGIS Geodatabase tiles into GRASS ? probably using v.in.ogr (PGeo driver [1]). Check available formats using `v.in.ogr -f`. If that fails I would also consider to ask the data provider to additionally publish the data in a documented data format. Markus [1] http://gdal.osgeo.org/ogr/drv_pgeo.html ___ grass-user mailing list grass-user@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-user
Re: re: [GRASS-user] Very high resolution topographic map of Europe: need help and advices: UPDATE
Dear All-who-may-be-intersted, First, I'd like to apologize for the delay: lots of stuff prevented me from working on the map. I've got some spared time now, and the map isn't finished yet - so I'm back into buisiness! Secondly, I'd like to thank you Markus, for the great script you sent me: it worked withouth a hitch, and I have now a single big shapefile to work on. And sorry for the last message: it was a misclick. Then comes the map: basically, even though the last replies provided my with some valuable advices, I'm still stuck with bathymetry - rivers and coastlines. 1. For the bathymetry, I finally took ETOPO1 data, which gives me another nice raster, although of much lower resolution (1' compared to the 3 topographic raster from SRTM DEM). So the plan is to cut off all the land data from the ETOPO1 raster along the coastlines, to keep only the sea aeras and than paste this reduced raster onto the main SRTM topographic raster of greater resolution. Can this step be automated ? (eg: is there a tool to do the job ?) Of course, should raster-merge be to complicated, there is always the other option, which is to export pngs first, and paste the pngs together; but in both cases, the bathymetric raster has to be cut. 2. Rivers and coastlines are a real pain. a. I could import a big pasted SWBD shapefile thanks to Markus script, but the cleaning process of the import command takes days, and gets somehow stuck within the brigdge removing phase. Result is an uncomplete vector map, lacking centroids, which can't be further processed by v.generalize for smoothing (the command dies saying there is an error in input data), but can be displayed with v.display. Another problem are all the square borders left around former SWBD tiles in water aeras; can they be removed ? Or do I have to manually edit the (huge) vector map ? b. Then come the rivers: the (somehow corrupted) SWBD vector map shows only coastlines, smaller closed waterbodies and - although only partially - larger rivers. I somehow have to complete the river data, and try following methods: -r.watershed command in grass, to compute the rivers from DEM data. This command just dies during the memory allocation process (KILL signal) because the map is too huge for my system. I tried with differend -m parameters, but the command always askes for up to 10gb virtual memory the kernel won't be able to provide. I guess I would have to work on smaller pieces of the map, set with g.region, and paste the results together, but this is another time consuming option. -VMAP0 data import works, but the result is quite disappointing. The secondary rivers network seems quite good, but this datased is unable to fix the uncomplete major rivers from the SWBD dataset. For an instance, by showing both layers (SWBD + VMAP0) on the monitor, I get all smaller rivers flowing into the rhine, while the rhine itself remains divided into several un-joined segments. -OSM(openstreetmap) data is a bit confusing. First it is divided along countries, so you have to dowload and paste lots of different files. Than the Waterway shapefile is quite poor. And finally the natural shapefile comes with nice river data, but also a lot of confusing data, like forest aeras. This need some sort of filtering, but I don't know at all how to do this. c. Of course, if someone knows better river datasets (scale approching 3, complete, easy to use), I would be happy to try them. Voila. Even though a lot more problems arise during each step (resolution has become another one, for an instance), I hope this project will see an end soon. Thanks for your help and your patience, Felix 2009/8/17 Markus Neteler nete...@osgeo.org: Hi Felix, On Sun, Aug 16, 2009 at 7:37 PM, Felix Schalckfelix.scha...@gmail.com wrote: ... a - Importing the vectors from SWBD is no problem, tough It would be nice to have the 200+ NASA shapefiles merged BEFORE importing a new layer in GRASS. Is this possible with ogr2ogr ? Merge of two SHAPE files 'file1.shp' and 'file2.shp' into a new file 'file_merged.shp' is performed like this: # note order out, then in: ogr2ogr file_merged.shp file1.shp ogr2ogr -update -append file_merged.shp file2.shp -nln file_merged file2 The second command is opening file_merged.shp in update mode, and trying to find existing layers and append the features being copied. The -nln option sets the name of the layer to be copied to. Attached a script to do as many as you want. b - The big problem are coastlines and waterbodies (+main rivers): somehow I have to show them on the topographic map, which gdalwarp has filled out with -32768 values in nodata-waterzones. So either I cut waterbodies out of the topographic raster along the vectors, or I somehow have GRASS compute me all waterbodies from the vector layer, fill them with a nice blue and create a raster which can be pasted over the topographic raster to get the final map. It seems doable with mapcalc, but frankly, I do not
Re: [GRASS-user] Low map display resolution in default 6.4 ?
2009/8/17 Moritz Lennert mlenn...@club.worldonline.be: On 16/08/09 21:48, Felix Schalck wrote: After some research, I figured out that: 1) Resolution is not really the problem, since I added a shadings layer which shows the same impressive details level than I used to have in old 6.23. It is really a display question. 2) Perhaps it is the graphic driver. I did not yet found the ./configure summary of the default ubuntu GRASS .deb, but I really doubt that it is built against latest proprietary nvidia 64bits OpenGL libs - which I did in my own compilation of 6.4RC5. Fact is that display has changed for all GRASS operations (not only d.rast), and even though the new display doesn't LOOK very sharp, the levels of details is just the same, but with an impressive gain in speed. What do you think ? What do you use to display you maps ? If you use the GUI, note that the GUI does not necessarily display in the current working resolution, but often downgrades resolution for display in order to make it faster. That's was exactly my question: so GUI (I use wxpython interface) DEOS resampling to fasten dispay operations ! However, you can constrain the display to your computational region's resolution. How do you do that ? You should see no difference in display if you use x-monitors (i.e. launched with d.mon. My memories are a bit weak of that time (long ago), but I think that the 6.2 GUI still used x-monitors for display. This has changed since. Moritz ___ grass-user mailing list grass-user@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-user
Re: re: [GRASS-user] Very high resolution topographic map of Europe: need help and advices: UPDATE
Dear Markus and other Grass users,
Here comes the latest update about my high resolution topographic map
of Europe - as usual - with some new questions.
Part1 (pasting cgiar-srtm tiles together) was a real pain, since I
needed to recompile GDAL with BigTIFF support in GTiFF driver to be
able to create such large geotiff files. Of course, once new gdal
successfully installed, I had to recompile GRASS against the new
gdal-bigtiff...
As posted on the gdal-dev mailing list, strange things occured during
my various gdalwarp attemps: while pasting all tiles bewteen
{34N;60N;-11W;35E} together in one command takes about 20h, dividing
the job in:
a - first merge slices of saying all tiles located in one long. column
b - finally paste all slices together
reduces the time needed to complete the command to around 4-5h. (I
posted 1 hour completion-time in the gdal list, but it was on a
smaller region)
Part2 has been updated to import the geotiff file in a lat/long
projection location native to srtm data, and than reproject the
obtained raster into my selected lcc (lambert conical conformal)
projection. This was probably the longest part, since it took r.proj
about 8h to complete this task. It tooks me actually twice as long,
since my region bounds were not properly set to include the distortion
of the lcc projection, so that the first re-projected raster had been
cut off just over Denmark; and I had to launch the reprojection
again...
Part3 is about creating the shadings, still ongoing, and works
surprinsingly well with r.shaded.relief, tough I'm still messing
around with zmult and azimut options to get the desired shadings.
Should somebody have advices here, to get nice shadings for large
srtm-based topographic maps, feel free to post them.
@Markus: I did not try the gdaldem tool to create the shadings, since
I re-projected the DEM in GRASS raster format.
Part4 is were I'm stuck currently, and is about creating coastline
vectors, bathymetry and exporting the results to be re-worked/merged
in GIMP.
a - Importing the vectors from SWBD is no problem, tough It would be
nice to have the 200+ NASA shapefiles merged BEFORE importing a new
layer in GRASS. Is this possible with ogr2ogr ?
b - The big problem are coastlines and waterbodies (+main rivers):
somehow I have to show them on the topographic map, which gdalwarp has
filled out with -32768 values in nodata-waterzones. So either I cut
waterbodies out of the topographic raster along the vectors, or I
somehow have GRASS compute me all waterbodies from the vector layer,
fill them with a nice blue and create a raster which can be pasted
over the topographic raster to get the final map. It seems doable
with mapcalc, but frankly, I do not know at all how to proceed. Any
help here would be greatly appreciated.
Part5 will be about secondary rivers, which will either be taken
directly from VMAP0 hydro layer or extracted with r.watershed (thank
you Markus for pointing this possibility) following the nice little
tutorial provided in the latest r.watershed manpage (bottom of the
page).
I will probably tryout both on a small scale, and keep the one
offering the best output (at my scale). The plan is to create another
vector layer showing only secondary rivers. More on this to follow.
Part6/final: rescaling (optional), exporting and merging in GIMP.
Notice that - at least until now, very little rework is required so
that I might consider finishing the job with GRASS and export one
finished map.
Voila. I sincerely hope this progress report is of some interest,
Thank you for your time and patience when it comes to answer my
numerous questions,
regards,
Felix
___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user
[GRASS-user] Transparent-background elevation shadings
Hi, Aside my previous - more complete - mail, here is one quick question: Is it possible to create an alpha-to-black color table in GRASS, which could than be assigned to an elevation shadings (created via r.relief.shaded) layer in order to get a transparent background on elevation layers ? Thank you for your time, Felix ___ grass-user mailing list grass-user@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-user
Re: [GRASS-user] Low map display resolution in default 6.4 ?
After some research, I figured out that: 1) Resolution is not really the problem, since I added a shadings layer which shows the same impressive details level than I used to have in old 6.23. It is really a display question. 2) Perhaps it is the graphic driver. I did not yet found the ./configure summary of the default ubuntu GRASS .deb, but I really doubt that it is built against latest proprietary nvidia 64bits OpenGL libs - which I did in my own compilation of 6.4RC5. Fact is that display has changed for all GRASS operations (not only d.rast), and even though the new display doesn't LOOK very sharp, the levels of details is just the same, but with an impressive gain in speed. What do you think ? Felix 2009/8/15 Hamish hamis...@yahoo.com: Felix wrote: Driver: GTiff/GeoTIFF Files: srtm_38_03.tif Size is 6000, 6000 2. Region settings: $g.region -p rows: 31200 cols: 55200 rows, cols to not match. try 'g.region rast=your_imported_raster_map' Hamish ___ grass-user mailing list grass-user@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-user
[GRASS-user] Low map display resolution in default 6.4 ?
Hello, Since I compiled latest 6.4RC5 Grass version, I experienced significative speed gain when displaying DEM raster levels - at the expense of the displayed resolution: the whole thing just looks soappy, and keeps getting worse once you zoom in. With my previous version (6.23), the same raster display opreration (d.rast) took forever, but I had the possibility to control data errors at the finest level. So surely there must be an option I don't know yet to increase the resolution of the Map display windows; it would be very nice if somebody could explain me how it works. Thanks for your help, Felix ___ grass-user mailing list grass-user@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-user
Re: [GRASS-user] Low map display resolution in default 6.4 ?
Dear Nikos, Many thanks for your reply. But even before listing up the requested data, please noticed that, basically, I didn't change any settings (g.region) when upgrading to 6.4: I just compiled and installed the new version, and now everthing I loaded in 6.23 looks soappy in 6.4. Resolution is important for both live display in GRASS, to control data elevation errors, and further processing, since the final pgn map should be exported at the highest possible resolution. As of the requested additionnal data, here you go: 1. DEM resolution: $gdalinfo srtm_38_03.tif Driver: GTiff/GeoTIFF Files: srtm_38_03.tif Size is 6000, 6000 Coordinate System is: GEOGCS[WGS 84, DATUM[WGS_1984, SPHEROID[WGS 84,6378137,298.2572235630016, AUTHORITY[EPSG,7030]], AUTHORITY[EPSG,6326]], PRIMEM[Greenwich,0], UNIT[degree,0.0174532925199433], AUTHORITY[EPSG,4326]] Origin = (5.000,50.000) Pixel Size = (0.0008333,-0.0008333) Metadata: AREA_OR_POINT=Area Image Structure Metadata: INTERLEAVE=BAND Corner Coordinates: Upper Left ( 5.000, 50.000) ( 5d 0'0.00E, 50d 0'0.00N) Lower Left ( 5.000, 45.000) ( 5d 0'0.00E, 45d 0'0.00N) Upper Right ( 10.000, 50.000) ( 10d 0'0.00E, 50d 0'0.00N) Lower Right ( 10.000, 45.000) ( 10d 0'0.00E, 45d 0'0.00N) Center ( 7.500, 47.500) ( 7d30'0.00E, 47d30'0.00N) Band 1 Block=6000x1 Type=Int16, ColorInterp=Gray NoData Value=-32768 2. Region settings: $g.region -p projection: 3 (Latitude-Longitude) zone: 0 datum: wgs84 ellipsoid: wgs84 north: 60N south: 34N west: 11W east: 35E nsres: 0:00:03 ewres: 0:00:03 rows: 31200 cols: 55200 cells: 172224 If I interpret those numbers correctly, the region is set to the theoretical max. resolution. Again, since I basically did not touch the configs during the upgrade procedure (GRASS 6.236.4RC5) I thought to myself it might just be a configuration change of the default driver used by d.rast: in the old version, it took forever to load, but the level of details SEEMED much higher, now it loads quasi instantly, but looks 'soappy'. What changed ? Regards, Felix 2009/8/14 Nikos Alexandris nikos.alexand...@felis.uni-freiburg.de: Felix Schalck: Since I compiled latest 6.4RC5 Grass version, I experienced significative speed gain when displaying DEM raster levels - at the expense of the displayed resolution: the whole thing just looks soappy, and keeps getting worse once you zoom in. With my previous version (6.23), the same raster display opreration (d.rast) took forever, but I had the possibility to control data errors at the finest level. So surely there must be an option I don't know yet to increase the resolution of the Map display windows; it would be very nice if somebody could explain me how it works. Hi Felix! What is the resolution of your DEM? Did you set it using g.region? Are you interested in (only) displaying or (in addition in) processing your DEM at high-resolution? . If you are new in GRASS-GIS then its worthy to read the intro(s) [1][2]. Perhaps you are looking info on how to set the resolution which is done with the module g.region[3]. ( Note that g.region != r.region ) Apologies if you are familiar with all this, Nikos --- [1] http://grass.osgeo.org/grass64/manuals/html64_user/helptext.html [2] http://grass.osgeo.org/grass64/manuals/html64_user/rasterintro.html [3] http://grass.osgeo.org/grass64/manuals/html64_user/g.region.html ___ grass-user mailing list grass-user@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-user
RE:re: [GRASS-user] Very high resolution topographic map of Europe: need help and advices
Dear Markus, Thank you s much for your response; it provided me with valuables hints - but also with new questions. First, following your avice, I finally decided to compile the new GRASS6.4rc5. Having an amd64 cpu, there war no binary available; so I had to do the job myself. Let's face it: manual configuration is a pain for a newcommer like me, but once the ./configure script shows the long awaited final recap, the make command - although quite long - runs withouth a hitch. I reminds me of good ol' freebsd days, although we had a lot more troubles finishing compilation without errors: great job guys ! But the resulat was worth it; I don't know if it is the work you've done for the last few years, or the gcc optiomization flags (or perhaps both of it), but the resulting grass64 runs like a fireball compared to my old 6.2 bin provided by the Ubuntu repos. Again: great job ! Only now I fully understand what you meant by it is so far the only convincing software for GIS number crunching. But let's go back to the topic: my high-resolution topographic map of Europe. Thanks to your advices, the production outline has changed to this: 1. Merge the cgiar TIFs (AND NOT THE GRASS RASTERS IF I GOT THIS CORRECTLY) thanks to gdalwrap command. In what projection does this command work ? Is it possible to wrap the TIF directlly in my lcc projection ? 2. Add image pyramids with gdaladdo ( I frankly do not underdand this one at all; what does it mean ?) 3. Shaded relief: I don't know your gdalhillshade command at all. Does it produce the same results as the r.shaded.relief I was planning to use ? Can you set the illumination angle with gdalhilshade ? 4. Re-gdaladdo for the shaded tif. 5. Import in GRASS and checkout results. If I'm right, I will have two layers at this point, one with the relief colors, one with the shadings. 6. Coastlines and Rivers. I was planning to use SWBD(coastlines and main rivers) and VLMAP0 Data (for the secondary rivers). Here again, you provide me with a complete new set of tools: r.external/r.terraflow/r.mapcalc/. What is the general idea behind ?I checked the man pages, but I don't really understand how to use them for my purpose. My plan was to import the shapefile into the right projection with rvin.ogr, and than export svg files for rework BEFORE joining the river layer and the topographic layers; but perhaps your way, once I understand it, is more efficient. 7. Export the shaded topography with r.out.png in one big png. Do I need two files (one containing the shadings, one with the relief colors) ? 8. Merge topography and hydrography layers in GIMP. Could you (or anyone interested) please have a look a this ? Again: thanks you very much for your time, regards, Felix For some time now, I'm following sort of a personal objective to create the most precise (=high resolution) topographic map of Europe allowed by my comp (Xubuntu 9.04, AMD64, 3Gb RAM, 300+ HD space). Starting from the very scratch, I had to learn about DEMs and GIS/maptools - and I'm still not confortable with all the technicy behind. Fact is that the best data set available to serve my purpose seems to be the cgiar interpolated srtm3 geodata (no license problem here, since it is aimed for pure personal use) which has to somehow be translated into a big jpeg or png file. Here you can use gdalwarp to merge all files into a big one. Enjoy wildcards to do that in one line: gdalwarp srtm_*.tif cgiar_srtm3_LL.tif A big file is created. Then don't miss to add image pyramids: gdaladdo srtm3_LL.tif 2 4 8 16 32 64 and you can open a file of several GB in no time with QGIS for example. I posted some GDAL tricks here: http://gfoss.blogspot.com/2008/06/gdal-raster-data-tips-and-tricks.html (if you want OGR for vectors, check: http://gfoss.blogspot.com/2008/06/ogr-vector-data-tips-and-tricks.html ) ... The big problem happened to be the river data, since GSHHS provides only limited amount and precision of side-rivers, which resulted in chains of straight lines scattered all over a giant map, even after vectorization: it was a no-go. No idea, I didn't try GMT so much. And then, a few days ago, I discovered nasa SWBD data and WMAP0, which seem to be of much higher resolution, linked to a nice GRASS GIS tutorial on the french wikipedia. I immedialely dug into this new software, quite complicated I must admit, but very powerful. We hope you took the GRASS 6.4 version - even if still called release candidate it is used by many in production. Myself, I am doing heavy computations with it and it is so far the only convincing software for GIS number crunching :-) I figured out how to import GeoTiff data into GRASS Raster files, Hint: from GRASS 6.4 onwards you can use r.external to avoid import but to just link an external file into GRASS. Nice when registering 30GB of orthophotos in a few minutes... For newcomers QGIS is a nice interface to GRASS, too, since it comes with
Re: re: [GRASS-user] Very high resolution topographic map of Europe: need help and advices
2009/8/12 Markus Neteler nete...@osgeo.org: On Wed, Aug 12, 2009 at 4:06 PM, Felix Schalckfelix.scha...@gmail.com wrote: Dear Markus, Thanks to your advices, the production outline has changed to this: 1. Merge the cgiar TIFs (AND NOT THE GRASS RASTERS IF I GOT THIS CORRECTLY) thanks to gdalwrap command. In what projection does this command work ? Is it possible to wrap the TIF directlly in my lcc projection ? I tried that yesterday and did NOT have luck. I would do it two-pass, even if it consumes twice as much space temporaneously: a) gdalwarp: all into one file, keeping the projection (mosaicking) b) gdalwarp: reproject to LCC (note that there are EU LCC and others). Use your preferred resampling method (gdalwarp offers several). Perhaps I got something wrong and you can do it in one step as well. I immediately tried this, and ran into following problem: $gdalwarp srtm_*.tif europe_all_srtmV4_cgiar_default.tif Creating output file that is 6P x 36000L. ERROR 6: A 6 pixels x 36000 lines x 1 bands Int16 image would be larger than 4GB but this is the largest size a TIFF can be. Creation failed. If I understand this correctly, I don't have a choice here, and must reproject the whole thing while pasting it. So I computed this command: $gdalwarp -s_srs epsg:4326 -t_srs +proj=lcc +lat_1=47 +lat_2=41 +lat_0=53 +lon_0=12 +x_0=22.0 +y_0=15.0 +ellps=WSG84 +units=m +no_defs -tr 93 93 -r bilinear srtm_*.tif europe_all_srtmV4_cgiar_93m_LCC.tif It seems to work (at least a 3.6Gb TIF file is created in the right directory), but takes FOREVER (eg. the 2 first tiles took about 50min, and I have 56 tiles to be merged...). Strange thing is that neither the cpu nor the RAM is being used at full extend. The r.patch tool provided by GRASS went much faster. Any idea here to speed up things ? Would lowering the resolution (186m would be an option) help ? 2. Add image pyramids with gdaladdo ( I frankly do not underdand this one at all; what does it mean ?) OK, it means that map copies at lower to much lower resolution are stored in the GeoTIFF (or different format which supports pyramids) file. When then opening with QGIS, Mapserver etc, it takes advantage of that and speeds up in an impressive way the visualization. Of course the file size increases. Example: I take world natural earth 250m which is huge; to cover Europe you need to download 4 tiles = 8 GB. I added pyramids with gdaladdo and now I am able to open these huge 4 tiles in no time in one step with QGIS. It's a convenience. Thank you very much for the clear explanation. Indeed, this seems a very nice option. 3. Shaded relief: I don't know your gdalhillshade command at all. Does it produce the same results as the r.shaded.relief I was planning to use ? Can you set the illumination angle with gdalhilshade ? Yes. I realise that it is called gdaldem now. http://trac.osgeo.org/gdal/browser/trunk/gdal/apps/gdaldem.cpp http://gdal.org/gdaldem.html 4. Re-gdaladdo for the shaded tif. yes. 5. Import in GRASS and checkout results. If I'm right, I will have two layers at this point, one with the relief colors, one with the shadings. Right (say, one is the relief [colors are optional], the other the shadings). You can use d.his or r.his to make a nice shaded colorized terrain map, something like this: http://grass.osgeo.org/grass60/screenshots/images/etopo2_grass_laea_9_48N_0E.jpg Nice one: leads to a lot of graphical ideas concerning the final map... 6. Coastlines and Rivers. I was planning to use SWBD(coastlines and main rivers) and VLMAP0 Data (for the secondary rivers). Here again, you provide me with a complete new set of tools: r.external/r.terraflow/r.mapcalc/. What is the general idea behind ? r.external you would use in step 5. Instead of r.in.gdal. r.terraflow/r.mapcalc you may forget since I didn't understand that you would take the rivers as vector maps (I thought you wanted to extract them from the DEM). Didn't even think such things were possible, but now that you mention it, what would be your advice ? Using wmap0 set - with its errors - to get the rivers, or try to extract them from the DEM ? Which one would produce the best results (=at 93 or 186m resolution) ? I checked the man pages, but I don't really understand how to use them for my purpose. My plan was to import the shapefile into the right projection with rvin.ogr, v.in.ogr (or v.external). and than export svg files for rework BEFORE joining the river layer and the topographic layers; but perhaps your way, once I understand it, is more efficient. Not sure (since I was confused :). 7. Export the shaded topography with r.out.png in one big png. Do I need two files (one containing the shadings, one with the relief colors) ? If you do the color shading in GRASS, then you need to export only one file. 8. Merge topography and hydrography layers in GIMP. Yes, sounds reasonable. Please consider
[GRASS-user] Very high resolution topographic map of Europe: need help and advices
Hello, For some time now, I'm following sort of a personal objective to create the most precise (=high resolution) topographic map of Europe allowed by my comp (Xubuntu 9.04, AMD64, 3Gb RAM, 300+ HD space). Starting from the very scratch, I had to learn about DEMs and GIS/maptools - and I'm still not confortable with all the technicy behind. Fact is that the best data set available to serve my purpose seems to be the cgiar interpolated srtm3 geodata (no license problem here, since it is aimed for pure personal use) which has to somehow be translated into a big jpeg or png file. I started with GMT, and tried to render the whole aera between 34N and 60N, 11W and 35E with grdimage into a 6000x8000 pix [I made slices of 3°x46°, which I pasted together]. Given the amount of data, GMT softs take forever, but it seems to be the fate of this project to hit the limits of my hardware... The big problem happened to be the river data, since GSHHS provides only limited amount and precision of side-rivers, which resulted in chains of straight lines scattered all over a giant map, even after vectorization: it was a no-go. And then, a few days ago, I discovered nasa SWBD data and WMAP0, which seem to be of much higher resolution, linked to a nice GRASS GIS tutorial on the french wikipedia. I immedialely dug into this new software, quite complicated I must admit, but very powerful. I figured out how to import GeoTiff data into GRASS Raster files, and how to display/export each one of them, but soon had to face new problems, especially when tried to reproject the data into LCC projection. So I decided to ask for help on this mailing list. My Current plan is to: 1. reproject the cgiar raster tiles OR one big merged raster into LCC projection (native srtm data seems to be a strange Mercator) 2. create elevation shadings 3. export a nice shaded topographic png 4. extract rivers/coast vectors from SWBD files 5. workout in Inkscape 6. paste the whole thing together in GIMP. The main problem right now seems to be the tiling of the topographic data. Each cgiar-cell (5°x5°) can be shown into a separate layer, but I'm unable to work them together. And it looks like most of the tools provided by GRASS assume the raster map is the size of the location (r.proj for an example). I tried r.patch but it produce wired results on top of giant files (I stopped when I hit the 2Gb limit), perhaps because the cgiar tiles do not exactly match my region (11W 35E 60N 34N). Is there a way to cut-off the overlapping tiles ? And than I'm not sure the r.patch way is the right one, since the single resulting file will become really big. I took the time to explain the whole thing with the hope to not only get help about my immediate raster division problem, but also about the big-picture, eg: is GRASS the right tools to do this (I installed GRASS because of the SWBD data tutorial, but it seems to me that swbd could also be plotted via psxy in GMT )? How would you GIS-gurus proceed to create a high resolution topographic map ? Is one big png a good idea, or would it be smarter to divide the continent into 4 or more parts, and render each one into a separate png ? Thanks you very much for your time, Felix Schalck ___ grass-user mailing list grass-user@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-user
