Re: [gdal-dev] define netcdf time dimension
GDAL cannot be used to manipulate the netcdf dimension definitions. You should use something specialized in dealing with netcdf dimensions such as cdo [1] or nco [2]. [1] https://code.zmaw.de/projects/cdo [2] http://nco.sourceforge.net/ On Tue, Apr 14, 2015 at 2:46 AM, Dominik Schneider dominik.schnei...@colorado.edu wrote: Hi - I have some .mdl files from IDL with .hdr header files that I’d like to convert to netcdf. The following code produces a netcdf file with a subdataset for each band in the mdl file. Is there any way to define the bands as the time dimension, in this case 4416 hourly timesteps? Or is there another tool that can convert this? gdal_translate -of NetCDF swe.mdl swe.nc Thanks Dominik ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] trouble with netCDF in C#?
Does this happen when you use gdal_translate to copy the netcdf file to another format (say gtiff)? I am not familiar with c# but there must be a way to compile gdal and you application to get the exact line of code that causes the crash? Etienne On Mon, Mar 16, 2015 at 8:57 AM, Robbie Price pri...@landcareresearch.co.nz wrote: I’m having trouble converting some netcdf (version 3) files to Kea files using the GDAL C# bindings. The problem is some form of memory leak caused by the netCDF code (from what I can establish through various tests). There are two issues: 1) A memory leak in this call (which I noticed, but can work around as I am only dealing with a small number of subdatasets) a. dsNetCDF = OSGeo.GDAL.Gdal.Open(strNetCDF_DSN, OSGeo.GDAL.Access .GA_ReadOnly); 2) A memory issue/leak pointer assignment causing a crash that I cannot trace the error reports as: Managed Debugging Assistant 'FatalExecutionEngineError' has detected a problem in 'O:\Projects\KEA_ReadWrite\bin\x64\Debug\KEA_ReadWrite.vshost.exe'. Additional Information: The runtime has encountered a fatal error. The address of the error was at 0xebc85654, on thread 0x16ec. The error code is 0xc005. This error may be a bug in the CLR or in the unsafe or non-verifiable portions of user code. Common sources of this bug include user marshaling errors for COM-interop or PInvoke, which may corrupt the stack. The error is consistent in nature but happens inconsistently in the code (between 1800 and 4000 bands) – here from another run: Managed Debugging Assistant 'FatalExecutionEngineError' has detected a problem in 'O:\Projects\KEA_ReadWrite\bin\x64\Debug\KEA_ReadWrite.vshost.exe'. Additional Information: The runtime has encountered a fatal error. The address of the error was at 0xebc85654, on thread 0x199c. The error code is 0xc005. This error may be a bug in the CLR or in the unsafe or non-verifiable portions of user code. Common sources of this bug include user marshaling errors for COM-interop or PInvoke, which may corrupt the stack. The error *MAY* be occurring more often only on lines handling string functions (?) Version Details Using MSVS 2010 (C#) and the following: http://www.gisinternals.com/query.html?content=filelistfile=release-1600-x64-gdal-mapserver.zip *gdal-200-1600-x64-core.msi* http://download2.gisinternals.com/sdk/Download.aspx?file=release-1600-x64-gdal-mapserver/gdal-200-1600-x64-core.msi 3/15/2015 4:13 AM 21556 kB Generic installer for the GDAL core components Basic scenario is converting from one file type to another but taking the meta info from the NetCDF and using as band name info in the Kea file. Input files have between 365 and 8000 bands but not large spatially (260*243). -- Please consider the environment before printing this email Warning: This electronic message together with any attachments is confidential. If you receive it in error: (i) you must not read, use, disclose, copy or retain it; (ii) please contact the sender immediately by reply email and then delete the emails. The views expressed in this email may not be those of Landcare Research New Zealand Limited. http://www.landcareresearch.co.nz ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] NetCDF driver improvements (including groups support)
Hi Julien
These improvements look really great!
Unfortunately I am on a long-term vacation and semi-retired from my GIS
work. So I am sorry but I cannot really help on testing and committing the
work...
That is why I did not respond sooner.
As the last official netcdf maintainer I should at least give some
comments.
Perhaps Even can test the patches and commit them, or someone could step in
as a new netcdf maintainer.
If it is possible, It would be best to separate these improvements into a
number of patches. In any case you should add your patch(es) to a number of
new gdal trac ticket(s) (or existing ticket if it fixes a bug). I did
significant improvements some time ago and I understand that it is hard to
separate these improvements in several patches, so I don't think it is
absolutely necessary to split them up - but it helps debugging any
regressions. Especially the groups support, I/O improvements and NASA
products should have independent patches and tickets.
Regarding 10) it would be best to maintain backwards compatibility, but if
the issue is only with 1D dataset I don't think it matters that much.
Cheers,
Etienne
On Thu, Feb 5, 2015 at 11:10 PM, Julien Demaria julien.dema...@acri-st.fr
wrote:
Hi GDAL team,
I've implemented several improvements to the NetCDF driver and I would
like to provide them to the community.
Main goal of the changes is to add full support of NetCDF-4 including
groups.
NetCDF-4 is the future format of ESA Sentinel-3 products (no groups) and
NASA Ocean Color team is switching their L2/L3 products to NetCDF-4 with
groups (VIIRS has already switched to the new format in December).
With the changes NASA L2 products geolocation is automatically handled as
geolocation arrays and can be reprojected using gdalwarp.
I validated with autotests that nothing is broken in tests netcdf.py
(excepting test 13 but see my point 5), netcdf_cf.py and hdf5.py, using
NetCDF-3 and 4 libraries.
I've also tested the new functionalities on various NetCDF-4 files.
I think the only possible regression could be for marginal cases where a
file was seen directly as a dataset and is now seen as multiple subdatasets
(for example if a file has only one var in the top group and has nested
groups containing variables), but I think this is not very common.
For the moment I have all these changes in local GIT separated commits on
the latest gdal-1.11 branch, let me know what changes you want and how can
I provide them.
Changes :
1) Implement full support for NetCDF-4 groups on reading:
- explore recursively all nested groups to create the subdatasets list
- subdatasets in nested groups use the /group1/group2/.../groupn/var
standard
NetCDF-4 convention, excepting for variables in the root group which
do not
have a leading slash for backward compatibility
- when accessing a subdataset using NETCDF:$file:$path, the leading
slash is optional
- global attributes of each nested group are also collected in the
GDAL dataset
metadata, using the same convention
/group1/group2/.../groupn/NC_GLOBAL#attr_name,
excepting for the root group which do not have a leading slash for
backward compatibility
- when searching for a variable containing auxiliary information on
the selected subdataset,
like coordinate variables or grid_mapping, we now also search in
parent groups (using NCDFResolveVar).
I now this is something not specified at this time in the CF
convention because CF does not know groups,
but it seems logical to me to support this: NetCDF-4 specifies that
dimensions of a group are
shared to its nested groups, so associated coordinate variables
could be defined as the same level of its
corresponding dimension.
- reference to coordinate variables using the coordinates attribute
support now also absolute paths,
this allow for example to specify coordinate variables located
outside the group of the selected variable
or its parents. Relative paths could be implemented if needed.
This feature is used to add support for new NASA Ocean Color L2
products.
2) Implement full read/write support for new NetCDF4 types NC_UBYTE,
NC_USHORT, NC_UINT and NC_STRING, only if NETCDF_HAS_NC4 is defined (and
only if format=NC4 for writing).
Support implemented for variables and attributes.
NC_STRING type is supported for reading (scalar and arrays) attributes
and is used for writing only for array attributes (scalar are still written
as NC_CHAR).
If NETCDF_HAS_NC4 is not defined or format!=NC4, NC_STRING array
attributes are written as a single NC_CHAR string using the GDAL
{v1,v2,...} convention.
Add missing support for NC_BYTE in CreateBandMetadata() and
NC_BYTE/SHORT in NCDFPut1DVar().
3) Add support for new NASA Ocean Color L2 products and ESA Sentinel-3 L1
or
L2 products which use the NetCDF-4 format (with groups for NASA, see
Re: [gdal-dev] Does gdal_calc.py not calculate arithmetic with in the same file?
I am not sure, but you can do the following to make sure: cp infile infile2 gdal_calc.py -A infile --A_band 1 -B infile2 --B_band 0 --outfile outfile --calc (A-B)/(A+B) Also, perhaps the extra comma in your command might be to blame -B infile, Etienne On Tue, Jul 29, 2014 at 3:46 AM, Cleo Drakos cleo21dra...@gmail.com wrote: Hi GDAL Developers, I have one GeoTIFF file with 2 bands. The data type is Byte. I want to calculate (Band2-Band1)/(Band2+Band1). But the following command provided a dark image: gdal_calc.py -A infile --A_band 1 -B infile, --B_band 0 --outfile outfile --calc (A-B)/(A+B) Does gdal_calc.py not calculate arithmetic with in the same file? Thanks for your time. cleo ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] Write support for MBTiles driver
Note that the licence used in the software (GNU GPL v3) is incompatible with GDAL, so it cannot be included in the GDAL source tree without the author's consent to use the code under another licence (such as the MIT/X licence used by GDAL). On Sat, Jul 19, 2014 at 10:15 AM, Nicu Tofan nicu.to...@gmail.com wrote: Thanks, I'll have a look, I see that Simon Thépot is mentioned as author and Mark Johnson adapted them. Nick 2014-07-19 15:47 GMT+03:00 Eli Adam ea...@co.lincoln.or.us: http://geopaparazzi.github.io/geopaparazzi/ has write support built in as part of caching. I think that Mark is the person who added the MBTiles write support to Geopaparazzi. Best regards, Eli On Fri, Jul 18, 2014 at 12:07 PM, Nicu Tofan nicu.to...@gmail.com wrote: Hello, I would like to draw your attention to this ticket created by me: http://trac.osgeo.org/gdal/ticket/5578 Here is its content: In version 1.11.0 gdal provides a read-only driver for MBTiles thanks to the work of Even Rouault. For the application that I'm working on I need to create MBTiles files and I would like to make my work part of gdal's driver. If there are other efforts in this direction I would love to help. References: MBTiles http://mbtiles.org/ MBTiles Specification https://github.com/mapbox/mbtiles-spec Implementations https://github.com/mapbox/mbtiles-spec/wiki/Implementations Driver Page on gdal http://www.gdal.org/frmt_mbtiles.html gdal2mb script https://github.com/developmentseed/gdal2mb MBTilesGenerator PHP library https://github.com/HTML24/mbtiles-generator MBUtil Python package https://github.com/klokantech/mbutil mbtiler.cpp from MapBox https://github.com/mapbox/mbtiler/blob/master/mbtiler.cpp ReaderWriterMBTiles.cpp from osgearth http://updraft.github.io/osgearth-doc/html/ReaderWriterMBTiles_8cpp_source.html Thanks, Nick ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] geo PDF with layers
this page suggests that gdal was updated to 1.10, perhaps not for you fedora version or respositories https://lists.fedoraproject.org/pipermail/package-announce/2014-March/130765.html On Thu, Jul 17, 2014 at 3:41 PM, Even Rouault even.roua...@mines-paris.org wrote: Le jeudi 17 juillet 2014 17:38:06, brad a écrit : I have gdal 1.9.2 on fedora 19, that appears to be the latest build available from fedora. I have a geo PDF, from US Topo which has several layers, including an image layer. I would like to convert this to tiff without the image layer. From what I can figure out, I need 1.10 to do this. Is that true? There isn't a repository for more a more up to date version is there? Yes, you need 1.10 or 1.11. You can always compile GDAL from source. Thanks. ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev -- Geospatial professional services http://even.rouault.free.fr/services.html ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] feeding input files into gdal_merge.py
this is not really related to gdal but to python... you should try using shell=False and adding the entire command in one argument instead of an array, but I'm not sure that the file globbing (*.tif) would work. On Wed, Jul 9, 2014 at 12:31 PM, bas smit baspys...@gmail.com wrote: While merging the files using gdal_merge.py the following code works: subprocess.call([sys.executable,gmerge,'-o','C:\\r.tif','-of','GTiff','D:\\a.tif','D:\\b.tif'],shell=True) However, when the input files are numerous, each file can not be inserted separately. In this case, the following code does not work: subprocess.call([sys.executable,gmerge,'-o','C:\\r.tif','-of','GTiff','D:\\*.tif'],shell=True) I did not know how to input the input files. Any idea is welcomed. ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] gdal 1.11 unable to read netcdf files
It would help if you tell use how you installed gdal-1.11. Also where did this gdal_formats.txt file come from? On Fri, Jun 27, 2014 at 11:10 AM, 00darki alex.jong0...@gmail.com wrote: Hi, I'm new on the mailing list, and to begin i want to apologize for my bad english skill. I hope you could help me, i've a problem with gdal to read netcdf (.nc) files. In gdal_formats.txt , i found the line : netCDF (rw+s): Network Common Data Format But if i run *gdalinfo.exe --formats* , netcdf driver doesn't appear. And if I try something on a netcdf file : *d:\GDAL\gdal-1-11\bin\gdal\appsgdalinfo.exe D:/GDAL/MapServer/ mnt0.nc ERROR 4: `D:/GDAL/MapServer/mnt0.nc' not recognised as a supported file format. gdalinfo failed - unable to open 'D:/GDAL/MapServer/mnt0.nc'.* and same thing with : *d:\GDAL\gdal-1-11\bin\gdal\appsgdalinfo.exe NETCDF:D:/GDAL/MapServer/mnt0.nc ERROR 4: `NETCDF:D:/GDAL/MapServer/mnt0.nc' does not exist in the file system, and is not recognised as a supported dataset name. gdalinfo failed - unable to open 'NETCDF:D:/GDAL/MapServer/mnt0.nc'.* But i've a file called gdal_netCDF.dll in gdal plugin and a file netcdf.dll in bin directory.. I don't understand why gdal couldn't read any netcdf file .. I'm using gdal version 1.11 and windows 7 x64. Thank you for any help :) -- View this message in context: http://osgeo-org.1560.x6.nabble.com/gdal-dev-gdal-1-11-unable-to-read-netcdf-files-tp5148406.html Sent from the GDAL - Dev mailing list archive at Nabble.com. ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] gdal 1.11 unable to read netcdf files
If these plugin files came from gisinternals and netcdf is not supported without moving the plugin files to another directory, then that is a bug in gisinternals. Perhaps Tamas can comment on this? Etienne On Fri, Jun 27, 2014 at 12:11 PM, 00darki alex.jong0...@gmail.com wrote: ..I found a solution, simply copy .dll files in a subdirectory called /gdalplugins in the /apps directory .. it seems to works fine (with an error on gdal_GEOR.dll) *d:\GDAL\gdal-1-11\bin\gdal\appsgdalinfo.exe D:/GDAL/MapServer/ mnt0.nc ERROR 1: Can't load requested DLL: d:\GDAL\gdal-1-11\bin\gdal\apps\gdalplugins\gdal_GEOR.dll 193: %1 nÆest pas une application Win32 valide. ERROR 1: Can't load requested DLL: d:\GDAL\gdal-1-11\bin\gdal\apps\gdalplugins\gdal_GEOR.dll 193: %1 nÆest pas une application Win32 valide. Warning 1: No UNIDATA NC_GLOBAL:Conventions attribute Driver: netCDF/Network Common Data Format Files: D:/GDAL/MapServer/mnt0.nc Size is 4321, 2161 Coordinate System is `' Metadata: NC_GLOBAL#latitude_max=90 NC_GLOBAL#latitude_min=-90 NC_GLOBAL#longitude_max=180 NC_GLOBAL#longitude_min=-180 profondeurs#desc=GRILLE profondeurs#name=Profondeur profondeurs#units=metre Corner Coordinates: Upper Left (0.0,0.0) Lower Left (0.0, 2161.0) Upper Right ( 4321.0,0.0) Lower Right ( 4321.0, 2161.0) Center ( 2160.5, 1080.5) Band 1 Block=4321x1 Type=Float32, ColorInterp=Undefined NoData Value=9.969209968386869e+036 Metadata: desc=GRILLE name=Profondeur NETCDF_VARNAME=profondeurs units=metre* Thank you for your answer :) Problem solved -- View this message in context: http://osgeo-org.1560.x6.nabble.com/gdal-dev-gdal-1-11-unable-to-read-netcdf-files-tp5148406p5148434.html Sent from the GDAL - Dev mailing list archive at Nabble.com. ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] RFC 46 GDAL/OGR unification adopted and commited
On Sat, May 24, 2014 at 4:54 PM, Even Rouault even.roua...@mines-paris.orgwrote: Le samedi 24 mai 2014 21:28:39, Etienne Tourigny a écrit : Hi Even, great work! Quick testing was fine and revealed no errors. I was able to compile it in QGIS master very easily. The only issue I encountered was related to GDALRegisterAll() and OGRAllRegister() registering all drivers (raster and vector), requiring to ignore the vector-only drivers in the raster-only related code in QGIS. Yeah, I've noticed a similar issue in PostGIS because they unload now all drivers by default except those in a white list, and they still needed a OGR driver. May I suggest a change that would improve backwards-compatibility and avoid unwanted issues? Perhaps these 2 functions should only register raster or vector drivers (as in GDAL 1.x), instead of all drivers, and a new function would register both raster and vector drivers. This would ensure old code behaves as previously, and ensure that no extra drivers are loaded and probed when only raster/vector drivers are needed. Just to be clearer here is how I would see it: GDALRegisterAll() - registers raster drivers (those with GDAL_DCAP_RASTER=YES) OGRAllRegister() - registers vector drivers (those with GDAL_DCAP_VECTOR=YES) GDALRegisterDrivers() - registers all drivers On the other habd, I imagine this may not be easy to implement and could cause some confusion in the future. That could be done, although I'm not sure it is really worth doing. There will be likely other changes in GDAL 2.0 development that will require adjustments in calling code. In your suggestion a driver that has a raster and vector face (like the re- unified PDF and PCIDSK done in RFC 46) should be listed in the two lists. Actually in QGIS case, the issue is likely not driver registration, but driver listing (in QGIS, both GDAL and OGR driver should be registered). So that wouldn't help. sort of - the side effect of registering all vector drivers was that vector drivers were listed, so I filtered them using the GDAL_DCAP_RASTER Also, probing optimization can instead be achieved using GDALOpenEx() specifying raster vs.vector drivers. GDALOpen() will only probe drivers that have raster capabilities. It is an alias of GDALOpenEx(filename, GDAL_OF_RASTER | update_flag ) right- got confused there anyway - if important issues come up it might be worth changing this, if not no biggie. Again congrats! Etienne Even -- Geospatial professional services http://even.rouault.free.fr/services.html ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] Motion: Adopt RFC 46 : GDAL/OGR unification
+1 from me On Mon, May 19, 2014 at 6:00 PM, Jukka Rahkonen jukka.rahko...@mmmtike.fiwrote: Even Rouault even.rouault at mines-paris.org writes: Hi, I think that the points raised in the discussion have been answered. I've just done a minor edit to the RFC text to mention RFC 36, as rightly suggested by Ivan. So: Motion : I move to adopt RFC 46: GDAL/OGR unification http://trac.osgeo.org/gdal/wiki/rfc46_gdal_ogr_unification Starting with my +1 +0 because RFC goes too deeply into the code for me but I trust that Even has a good intention. -Jukka Rahkonen- ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] Call for discussion on RFC 46: GDAL/OGR unification
From my understanding of the RFC, legacy functions GDALOpen() would only try to open the dataset using raster drivers and OGROpen() would only try using vector drivers, so it shouldn't take more time than in gdal 1.x. It also seems that this RFC addresses the need to specify the driver(s) used to open the dataset. This effectively solves the problems raised in RFC 36, as Even mentions. In my view no need to work on RFC 36 as this should cover it (as long as the utilities are updated). Looks good to me! cheers, Etienne On Fri, May 16, 2014 at 8:36 AM, Even Rouault even.roua...@mines-paris.orgwrote: Ivan, yes indeed, I missed that one. I'll update the RFC with it. I think that the intent of RFC36 is covered by what is already proposed in RFC46 with the papszAllowedDrivers of GDALOpenEx(). I do not think that specifying the candidate driver(s) is that usefull in utilities since the time to launch them will be generally higher than the probing time. But that could be easily implemented later with an option if that was really needed (e.g. gdalinfo foo.tif -driver gtiff or possibly gdalinfo foo.jp2 -driver jp2ecw,jp2openjpeg). Due to the work I did in OGR drivers to avoid repeated file access by using GDALOpenInfo*, I don't think that GDAL drivers should be affected. And the OGR drivers are registered after the GDAL ones. So if the file being opened is really a GDAL dataset, the OGR drivers should have 0 impact on the opening time. I did try to benchmark a bit that, but it is difficult to come to a firm conclusion since there are many factors: hot/cold runs, whether you spawn a new process or use an existing one, etc.. And the time being measured are a few tens of milliseconds, so very sensitive to task scheduling. Even Even, I think that RFC 36 should be included in your list of related RFCs and should be reconsider for adoption since the unification would make the probing even more expensive. Regards, Ivan From: even.roua...@mines-paris.org To: gdal-dev@lists.osgeo.org Date: Thu, 15 May 2014 20:47:32 +0200 Subject: Re: [gdal-dev] Call for discussion on RFC 46: GDAL/OGR unification Le jeudi 08 mai 2014 00:13:22, Even Rouault a écrit : Hi, This is a call for discussion on RFC 46: GDAL/OGR unification http://trac.osgeo.org/gdal/wiki/rfc46_gdal_ogr_unification No reaction : no interest or no time to review yet ? Or should I move that forward ? But I'd prefer if such architectural changes could be a bit reviewed... Best regards, Even -- Geospatial professional services http://even.rouault.free.fr/services.html ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] Call for discussion on RFC 46: GDAL/OGR unification
With this RFC, you could achieve this using GDALOpenEx() e.g. GDALOpenEx( ASDFG, GDAL_OF_ALL, QWERTY) On Fri, May 16, 2014 at 11:01 AM, Ivan Lucena lucena_i...@hotmail.comwrote: Even, Yes, it is hard to measure the impact of probing in a single file or any interactive command line operations. In the current API the class Driver doesn't expose the Open method: *Driver drv = gdal.GetDriverByName(QWERTY);* *Dataset dst = drv.Open(ASDFG);* IMHO that is a *missing functionality* in GDAL API. GDAL API is a library to support Geospatial Data Abastration not just a format conversion tool. You should be able the use it even if your sole intention is do deal with the fictitious QWERTY format. IMHO. Yes, you can delete all the driver or put then all on the black list but that may not be the best solution. The frustration of some user/programmer could be that they know what driver they want to use and they don't want to waste time probing; they don't want GDAL to get confused by files that can be supported by driver A and B; and most important they want to process a very very large number of files (ETL) or repeated times (WEB). The RFC 36 is just a band-aid solution. It starts from the fact that some drivers already haven a prefix like HDF4: and use it to select the driver. As you can see, the suggested code is not that pretty [ http://trac.osgeo.org/gdal/ticket/3043] since there is no Driver::Open method available. Regards, Ivan Date: Fri, 16 May 2014 13:36:08 +0200 From: even.roua...@mines-paris.org To: lucena_i...@hotmail.com CC: even.roua...@mines-paris.org; gdal-dev@lists.osgeo.org Subject: RE: [gdal-dev] Call for discussion on RFC 46: GDAL/OGR unification Ivan, yes indeed, I missed that one. I'll update the RFC with it. I think that the intent of RFC36 is covered by what is already proposed in RFC46 with the papszAllowedDrivers of GDALOpenEx(). I do not think that specifying the candidate driver(s) is that usefull in utilities since the time to launch them will be generally higher than the probing time. But that could be easily implemented later with an option if that was really needed (e.g. gdalinfo foo.tif -driver gtiff or possibly gdalinfo foo.jp2 -driver jp2ecw,jp2openjpeg). Due to the work I did in OGR drivers to avoid repeated file access by using GDALOpenInfo*, I don't think that GDAL drivers should be affected. And the OGR drivers are registered after the GDAL ones. So if the file being opened is really a GDAL dataset, the OGR drivers should have 0 impact on the opening time. I did try to benchmark a bit that, but it is difficult to come to a firm conclusion since there are many factors: hot/cold runs, whether you spawn a new process or use an existing one, etc.. And the time being measured are a few tens of milliseconds, so very sensitive to task scheduling. Even Even, I think that RFC 36 should be included in your list of related RFCs and should be reconsider for adoption since the unification would make the probing even more expensive. Regards, Ivan From: even.roua...@mines-paris.org To: gdal-dev@lists.osgeo.org Date: Thu, 15 May 2014 20:47:32 +0200 Subject: Re: [gdal-dev] Call for discussion on RFC 46: GDAL/OGR unification Le jeudi 08 mai 2014 00:13:22, Even Rouault a écrit : Hi, This is a call for discussion on RFC 46: GDAL/OGR unification http://trac.osgeo.org/gdal/wiki/rfc46_gdal_ogr_unification No reaction : no interest or no time to review yet ? Or should I move that forward ? But I'd prefer if such architectural changes could be a bit reviewed... Best regards, Even -- Geospatial professional services http://even.rouault.free.fr/services.html ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] Update of EPSG database to v8.4
You might be able to gzip compress the file GDAL can read gzip files transparently, but I am not sure that the csv reading code in GDAL works with compressed files. On Thu, May 15, 2014 at 9:00 AM, Yann Chemin yche...@gmail.com wrote: Hi Even, I would be interested to include it. But I see that the 1Mb uncompressed text is an issue. If it was OK for the additional Mb in the source code, what would be the place to look into gdal code to create the patch needed? Thanks, Yann PS: did not copy other lists as I am not a member, registered to proj ML today though. On 15/05/2014, Even Rouault even.roua...@mines-paris.org wrote: Selon Yann Chemin yche...@gmail.com: Hi, is there planetary datum support in this new version (i.e. Moon 2000, or etc.)? No, I don't think that the EPSG folks are interested in other planets yet ;-) But Frank mentionned some time ago ( http://osgeo-org.1560.x6.nabble.com/IAU-2000-Coordinate-System-Dictionary-td3750798.html ) that he wanted to investigate how to deliver the IAU set of planetary coordinate systems , but this hasn't been pursued further AFAIK. I see there's a version of this file here : http://svn.osgeo.org/metacrs/sr.org/srsbrowser/data/IAU2000.wkt Even Yann On 14/05/2014, Even Rouault even.roua...@mines-paris.org wrote: Hi, I've followed the update process of the EPSG SRS database to latest v8.4, and just committed the updated files into libgeotiff, GDAL and PROJ trunk. Also submitted to PostGIS. From what I can see, among many changes and additions, 2 new projection methods have been added: * 1051,Lambert Conic Conformal (2SP Michigan) -- looks very close to standard Lambert Conic Conformal (2SP), with the addition of a new parameter K, the ellipsoid scaling factor. * 1052,Colombia Urban I don't think any of those 2 new methods are currently handled by PROJ, so the new PCS based on those projection methods (3 Michigan SRS, and several dozains of Colombia PCS) will not be handled by GDAL nor PROJ. Regarding GDAL, I've improved the add_esri_column.py script (see http://trac.osgeo.org/geotiff/changeset/2446), so that we have more ESRI DATUM names. That should make morphing from/to ESRI .prj files a bit better. Regarding PROJ.4 'espg' and PostGIS spatial_ref_sys.sql files, I've also added the list of SRS that are GEOCCS (GeoCentric) and COMPD_CS (Compound Horizontal + Vertical). Please let me know if you see issues. libgeotiff: http://trac.osgeo.org/geotiff/ticket/63 http://trac.osgeo.org/geotiff/changeset/2447 GDAL: http://trac.osgeo.org/gdal/ticket/5462 http://trac.osgeo.org/gdal/changeset/27322 PROJ: http://trac.osgeo.org/proj/ticket/236 http://trac.osgeo.org/proj/changeset/2448 Submitted for PostGIS : http://trac.osgeo.org/postgis/ticket/2737 Best regards, Even -- Geospatial professional services http://even.rouault.free.fr/services.html ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev -- -- ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] Gdal-Fortran-Linux Problem
pretty interesting - it would be nice to incorporate the gdal fortran bindings inside the gdal source! however, it is distributed under the GNU Lesser General Public License, so I think it's not possible... Cheers Etienne On Tue, May 13, 2014 at 10:23 AM, Fabian Niggemann niggem...@vista-geo.dewrote: Hi, I´m trying to link gdal to fortrangis to use the gdal fortran interface on Ubuntu 12.04. I´ve installed gdal Version 1.11. and installed the fortrangis package with disabled shared libraries. If I try now to compile a fortran program(e.g. Command: ifort -I */usr/local/include/* gdal_test.F90 /usr/local/lib/libfortranc.a /usr/local/lib/libshp.a /usr/local/lib/libfortrangis.a /usr/local/lib/libgdal.a) I get a lot of error messages stating undefined references: /usr/local/lib/libgdal.a(genbindataset.o): In function `_Alloc_hider': /usr/include/c++/4.7/bits/basic_string.h:272: undefined reference to `std::string::_Rep::_S_empty_rep_storage' /usr/local/lib/libgdal.a(genbindataset.o): In function `std::string::_Rep::_M_dispose(std::allocatorchar const)': /usr/include/c++/4.7/bits/basic_string.h:237: undefined reference to `std::string::_Rep::_S_empty_rep_storage' /usr/include/c++/4.7/bits/basic_string.h:246: undefined reference to `std::string::_Rep::_M_destroy(std::allocatorchar const)' /usr/local/lib/libgdal.a(genbindataset.o): In function `GDALRegister_GenBin': ... Any help would be appreciated. Regards, Fabian -- VISTA Geowissenschaftliche Fernerkundung GmbH Administration: Anton-Ferstl-Str.11, D-82234 Wessling Office: Gabelsbergerstr.51, D-80333 München Tel. +49 89 287 794 59 Fax +49 89 523 89 804 eMail: niggem...@vista-geo.de; web: www.vista-geo.de Amtsgericht München HRB 129776 Sitz der Gesellschaft: Wessling Geschäftsführung: Dr. Heike Bach Bankverbindung: Hypo Vereinsbank München BLZ 70020270 Konto 39042770 USt-IDNr. DE812904252 St.Nr. 11714150105 ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] Idea: GeoTIFF box in JPEG to add georeferencing
Right, I didn't think of the need to create new EXIF tags.
I think it is a good idea to use existing GTIFF tags to encode this
information.
On the other hand, the method used in jpeg files should ideally be the same
used in other formats (e.g. png, gif) for which it is possible to add extra
metadata.
In light of this, it may be better to use an xml or textual representation
and embed it inside an XMP block, which is supported for many formats[1].
Also it would allow for easier human-reading.
[1]
http://en.wikipedia.org/wiki/Extensible_Metadata_Platform#Embedding_of_XMP
On Sat, May 10, 2014 at 6:49 PM, Even Rouault
even.roua...@mines-paris.orgwrote:
Le samedi 10 mai 2014 23:31:30, Etienne Tourigny a écrit :
Hi Even,
why not simply add the geotransform and WKT or PROJ.4 string as exif
tags?
simple to read outside of gdal (any exif reader will support this) and
implement using libexif.
You would have to create new EXIF tags for that. The advantage of using
GeoTIFF is that it is already standardized. And it has a few bonus, like
ground control points.
just my 2 cents.
Etienne
On Sat, May 10, 2014 at 5:21 PM, Even Rouault
even.roua...@mines-paris.orgwrote:
Hi,
I'm wondering if there's some interest in standardizing a way of
embedding georeferencing information in popular image formats like JPEG
or PNG.
I've just been doing an experiment with JPEG. My solution is to use a
GeoTIFF
box as specified in § 2.2 Box contents of
http://www.lizardtech.com/download/geo/geotiff_box.txt , a.k.a GeoJP2
and
heavily used to add georeferencing to JPEG2000 images.
The GeoTIFF file is put in an JFIF APP1 segment with a GeoTIFFBox\0
signature
to begin the payload.
So the structure is the following :
\xFF \D8 -- SOI marker (Start Of Image)
[ Any optional JPEG segment ]
\xFF \xE1-- APP1 marker
Size-- Number of byte of signature (11) +
size of {geotiff-file}.
little-endian
uint16
GeoTIFFBox\0 -- Signature (to distinguish from Exif, etc...)
{geotiff-file} -- A GeoTIFF image of size 1x1 (cf § 2.2 Box
contents)
[ Any optional JPEG segment ]
[ JPEG baseline segments ]
\xFF \D9 -- EOI marker (End Of Image)
End of spec !
Such a solution should have a good compatibility level, since the
GeoTIFFBox
APP1 segment should be ignored gracefully by JPEG readers not aware of
the spec. My few tests with various readers would confirm that.
A working implementation based on GDAL trunk can be found here :
https://github.com/rouault/gdal2/tree/geotiffbox_in_jpeg
The patch is :
https://github.com/rouault/gdal2/compare/geotiffbox_in_jpeg
Example of such a file attached to this email. Generated by :
gdal_translate
http://svn.osgeo.org/gdal/trunk/autotest/gcore/data/byte.tif
byte.jpg -of jpeg
And
$ gdalinfo byte.jpg
Driver: JPEG/JPEG JFIF
Files: byte.jpg
Size is 20, 20
Coordinate System is:
PROJCS[NAD27 / UTM zone 11N,
GEOGCS[NAD27,
DATUM[North_American_Datum_1927,
SPHEROID[Clarke 1866,6378206.4,294.9786982139006,
AUTHORITY[EPSG,7008]],
AUTHORITY[EPSG,6267]],
PRIMEM[Greenwich,0],
UNIT[degree,0.0174532925199433],
AUTHORITY[EPSG,4267]],
PROJECTION[Transverse_Mercator],
PARAMETER[latitude_of_origin,0],
PARAMETER[central_meridian,-117],
PARAMETER[scale_factor,0.9996],
PARAMETER[false_easting,50],
PARAMETER[false_northing,0],
UNIT[metre,1,
AUTHORITY[EPSG,9001]],
AUTHORITY[EPSG,26711]]
Origin = (440720.000,3751320.000)
Pixel Size = (60.000,-60.000)
Metadata:
AREA_OR_POINT=Area
Corner Coordinates:
Upper Left ( 440720.000, 3751320.000) (117d38'28.21W, 33d54' 8.47N)
Lower Left ( 440720.000, 3750120.000) (117d38'27.92W, 33d53'29.51N)
Upper Right ( 441920.000, 3751320.000) (117d37'41.48W, 33d54' 8.71N)
Lower Right ( 441920.000, 3750120.000) (117d37'41.20W, 33d53'29.75N)
Center ( 441320.000, 3750720.000) (117d38' 4.70W, 33d53'49.11N)
Band 1 Block=20x1 Type=Byte, ColorInterp=Gray
Image Structure Metadata:
COMPRESSION=JPEG
There are many potential other solutions, like adding the GeoTIFF tags
in
a proper EXIF segment, since it is already a TIFF structure, but that
would have
made my implementation a bit harder, and I'm wondering how some EXIF
readers
would react when facing GeoTIFF tags. Another solution would be to use
the heavy GMLJP2 specification (
http://www.opengeospatial.org/standards/gmljp2 ),
which is an OGC standard, but it is still evolving (
http
Re: [gdal-dev] Idea: GeoTIFF box in JPEG to add georeferencing
Hi Even,
why not simply add the geotransform and WKT or PROJ.4 string as exif tags?
simple to read outside of gdal (any exif reader will support this) and
implement using libexif.
just my 2 cents.
Etienne
On Sat, May 10, 2014 at 5:21 PM, Even Rouault
even.roua...@mines-paris.orgwrote:
Hi,
I'm wondering if there's some interest in standardizing a way of embedding
georeferencing information in popular image formats like JPEG or PNG.
I've just been doing an experiment with JPEG. My solution is to use a
GeoTIFF
box as specified in § 2.2 Box contents of
http://www.lizardtech.com/download/geo/geotiff_box.txt , a.k.a GeoJP2 and
heavily used to add georeferencing to JPEG2000 images.
The GeoTIFF file is put in an JFIF APP1 segment with a GeoTIFFBox\0
signature
to begin the payload.
So the structure is the following :
\xFF \D8 -- SOI marker (Start Of Image)
[ Any optional JPEG segment ]
\xFF \xE1-- APP1 marker
Size-- Number of byte of signature (11) +
size of {geotiff-file}. little-endian
uint16
GeoTIFFBox\0 -- Signature (to distinguish from Exif, etc...)
{geotiff-file} -- A GeoTIFF image of size 1x1 (cf § 2.2 Box
contents)
[ Any optional JPEG segment ]
[ JPEG baseline segments ]
\xFF \D9 -- EOI marker (End Of Image)
End of spec !
Such a solution should have a good compatibility level, since the
GeoTIFFBox
APP1 segment should be ignored gracefully by JPEG readers not aware of the
spec. My few tests with various readers would confirm that.
A working implementation based on GDAL trunk can be found here :
https://github.com/rouault/gdal2/tree/geotiffbox_in_jpeg
The patch is :
https://github.com/rouault/gdal2/compare/geotiffbox_in_jpeg
Example of such a file attached to this email. Generated by :
gdal_translate
http://svn.osgeo.org/gdal/trunk/autotest/gcore/data/byte.tif
byte.jpg -of jpeg
And
$ gdalinfo byte.jpg
Driver: JPEG/JPEG JFIF
Files: byte.jpg
Size is 20, 20
Coordinate System is:
PROJCS[NAD27 / UTM zone 11N,
GEOGCS[NAD27,
DATUM[North_American_Datum_1927,
SPHEROID[Clarke 1866,6378206.4,294.9786982139006,
AUTHORITY[EPSG,7008]],
AUTHORITY[EPSG,6267]],
PRIMEM[Greenwich,0],
UNIT[degree,0.0174532925199433],
AUTHORITY[EPSG,4267]],
PROJECTION[Transverse_Mercator],
PARAMETER[latitude_of_origin,0],
PARAMETER[central_meridian,-117],
PARAMETER[scale_factor,0.9996],
PARAMETER[false_easting,50],
PARAMETER[false_northing,0],
UNIT[metre,1,
AUTHORITY[EPSG,9001]],
AUTHORITY[EPSG,26711]]
Origin = (440720.000,3751320.000)
Pixel Size = (60.000,-60.000)
Metadata:
AREA_OR_POINT=Area
Corner Coordinates:
Upper Left ( 440720.000, 3751320.000) (117d38'28.21W, 33d54' 8.47N)
Lower Left ( 440720.000, 3750120.000) (117d38'27.92W, 33d53'29.51N)
Upper Right ( 441920.000, 3751320.000) (117d37'41.48W, 33d54' 8.71N)
Lower Right ( 441920.000, 3750120.000) (117d37'41.20W, 33d53'29.75N)
Center ( 441320.000, 3750720.000) (117d38' 4.70W, 33d53'49.11N)
Band 1 Block=20x1 Type=Byte, ColorInterp=Gray
Image Structure Metadata:
COMPRESSION=JPEG
There are many potential other solutions, like adding the GeoTIFF tags in a
proper EXIF segment, since it is already a TIFF structure, but that would
have
made my implementation a bit harder, and I'm wondering how some EXIF
readers
would react when facing GeoTIFF tags. Another solution would be to use the
heavy GMLJP2 specification (
http://www.opengeospatial.org/standards/gmljp2 ),
which is an OGC standard, but it is still evolving (
http://www.opengeospatial.org/standards/requests/118 ), and is rather
verbose
which defeats the purpose of JPEG being a compressed format...
Any opinion/interest ?
Even
--
Geospatial professional services
http://even.rouault.free.fr/services.html
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] accessing netCDF files with /vsicurl fails
Even is right - it never worked with the netcdf driver. But - it works if you use the vsipreload mechanism e.g. http://lists.osgeo.org/pipermail/gdal-dev/2013-May/036359.html On Sun, Apr 27, 2014 at 6:03 PM, Joaquim Luis jl...@ualg.pt wrote: On 27-04-2014 21:25, Even Rouault wrote: Le dimanche 27 avril 2014 21:50:36, Joaquim Luis a écrit : Hi, This type of command used to work before, but it now errors (fresh Win7 64 GDAL) Joaquim, I doubt it has ever worked with netCDF. The netCDF driver doesn't support the VSI Virtual file system API, since the I/O is done by the netCDF library itself. hm, I would be willing to swear ... but better not Thanks ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] problem with gdal_translate jp2 file from the usgs (linux)
On Thu, Apr 24, 2014 at 6:43 PM, Eli Adam ea...@co.lincoln.or.us wrote: Kasper, On Thu, Apr 24, 2014 at 2:32 PM, Even Rouault even.roua...@mines-paris.org wrote: Le jeudi 24 avril 2014 23:29:03, Kasper van Wijk a écrit : thank you, Even, for the fast reply. I am a beginner at GDAL and installed the gdal suite with standard apt-get and yum, respectively. Can you point me to a place where I can read how to compile GDAL against one of the other JPEG2000 libraries, as you say? Compiling is a great option and might be the best. You might also want to check which drivers are in Ubuntu GIS and Enterprise Linux GIS which might be easier than compiling. They are often more recent and with more features than the standard repos. https://wiki.ubuntu.com/UbuntuGIS http://wiki.osgeo.org/wiki/Enterprise_Linux_GIS Looks like it hasn't been updated in quite a while (a full year actually), it's using gdal 1.9.2 and qgis 1.8 From their page: The goal of the Enterprise Linux GIS (ELGIS) effort is to ensure that the latest stable versions of the main free GIS software run on the Enterprise Linux platform. cheers, Etienne Best regards, Eli You will find various resources at http://trac.osgeo.org/gdal/wiki/BuildHints Most interesting links for your use case : * http://trac.osgeo.org/gdal/wiki/DownloadSource * http://trac.osgeo.org/gdal/wiki/BuildingOnUnix * http://trac.osgeo.org/gdal/wiki/ECW : if you want to compile against ECW SDK * http://gdal.org/frmt_jp2openjpeg.html : if you want to compile against OpenJPEG v2 Thanks again, Kasper On 25 April 2014 09:25, Even Rouault even.roua...@mines-paris.org wrote: Le jeudi 24 avril 2014 23:17:29, Kasper van Wijk a écrit : dear all, I am downloading jp2 data from the usgs national map. To plot this in cartopy, I need to transform to something else, such as geotiff. However, using gdal_translate I get a warning copied below, and the output is a grainy bw version of the image. I have checked my results under linux both fedora core 20 and ubuntu 12.04 versions with GDAL 1.7.3, released 2010/11/10 gdal_translate m_3810623_ne_13_1_20110921_20111031.jp2 test.tiff -of GTiff warning: not enough tile data (2 bytes) Apparently your GDAL build uses libjasper as the jpeg2000 library. Jasper doesn't work very well unfortunately with some real world images. You might want to compile GDAL against one of the other JPEG2000 libraries that are supported : see http://gdal.org/formats_list.html -- Geospatial professional services http://even.rouault.free.fr/services.html -- Geospatial professional services http://even.rouault.free.fr/services.html ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] How to use OGROpen with WFS?
you probably need to include the entire string, including the WFS: prefix
On Mon, Apr 14, 2014 at 10:53 AM, David Tran david.t...@hsr.ch wrote:
Hi
I tried this:
OGRRegisterAll();
sourceName = http://maps.zh.ch/wfs/FnsNSWFS;;
sourceData = OGROpen( sourceName.c_str(), 0, NULL );
if(sourceData == NULL) {
printf(Open failed.\n);
exit(1);
}
and it ends with:
ERROR 4: Failed to read GeoJSON data
Open failed.
Does anybody has a proper example?
ogrinfo -ro -so WFS:http://maps.zh.ch/wfs/FnsNSWFS; shows something,
therefore I think that it is my source code.
Thanks in advance.
David
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] Problems with large ASTER GeoTIFF
On Mon, Apr 14, 2014 at 11:16 AM, Nicole Stoffels stoff...@f2e.de wrote: Dear Gdal-List, I have a large ASTER file (downloaded from http://www.eea.europa.eu/data- and-maps/data/eu-dem#tab-european-data) in the GeoTIFF-format. It is compressed with LZW. What I want to do is: 1. unpack it 2. seperate a smaller region from it 3. convert this regional GeoTIFF to a PNG What I am doing: $ gdal_translate -co COMPRESS=NONE EUD_CP-DEMS_6500035000-AA.tif tmp1.tif $ gdal_translate -srcwin 6500 3300 1000 1000 tmp1.tif tmp2.tif $ gdal_translate -of PNG tmp2.tif tmp2.png When I display the info with gdalinfo the first step seems to produce a decompressed file. The line COMPRESSION=LZW is not there anymore. You need to set the compression options on any gdal command, it does not use the compression of the input file. Also, you specifically asked for no compression, so it's doing what you told it to. use -co COMPRESS=LZW However, if I want to display the min and max with gdalinfo, I get the following error. ERROR 1: Failed to compute min/max, no valid pixels found in sampling. try without changing the extents and see if a single command (as suggested by Jean-Luc) works I thought that the file might be too large, but if I change the data_type from float 32 to e.g. Int16 I get 0 as min and max value. That's why I assume that already the first gdal_translate does not work as I actually want it to work. your input values are probably between 0 and 1 in this case The other two transformations work without error messages. However, in the end I get a black png. I have tried all types of transformation with various options. But as I am new in the subject and do not know much about gdal, I have no further ideas how to generate a file from my large GeoTIFF to continue working with. I hope someone has an idea, what's going wrong with my gdal_translations. Thanks very much in advance. Best regards Nicole ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] gdalwarp bug using -r mode
Hi, If there is a bug I am responsible for it... I have just updated the ticket at http://trac.osgeo.org/gdal/ticket/5433 Etienne On Tue, Apr 8, 2014 at 3:03 PM, Jonathan Greenberg j...@illinois.eduwrote: Folks: I'm running a gdalwarp on a sinusoidal MODIS class image to longlat, using a modal resample (and cropping the output). The warp is clipping off a circular region of North America and Australia. I am using multithreaded mode, but I confirmed this error also occurs in sequential mode. The error occurs on both Windows and Linux boxes running GDAL 1.10.1. The error does NOT occur using e.g. a nearest neighbor resample. gdalwarp -multi -te -180 -59.944 180 88.556 -ts 720 297 -t_srs +proj=longlat +datum=WGS84 +no_defs -r mode -of GTiff -wo NUM_THREADS=ALL_CPUS MOD44B_V5_TRE.2001_fixed_10k.envi MCD12Q1.A2001001_10k_05deg.tif I dropped the input and the output files in: https://drive.google.com/folderview?id=0B8Kij67bij_AVG1fWko4bFdTRFkusp=sharing Any ideas on how to solve this? I'll also submit a bug using TRAC, but I wanted to see if there were any short-term fixes for this error. --j -- Jonathan A. Greenberg, PhD Assistant Professor Global Environmental Analysis and Remote Sensing (GEARS) Laboratory Department of Geography and Geographic Information Science University of Illinois at Urbana-Champaign 259 Computing Applications Building, MC-150 605 East Springfield Avenue Champaign, IL 61820-6371 Phone: 217-300-1924 http://www.geog.illinois.edu/~jgrn/ AIM: jgrn307, MSN: jgrn...@hotmail.com, Gchat: jgrn307, Skype: jgrn3007 ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
[gdal-dev] errors in GDAL TMS examples
Hi, I have been looking at the GDAL TMS examples in [1] and it seems there are a few errors [1] http://www.gdal.org/frmt_wms.html I am not sure if the driver, server path or server changes are to blame. 1) shouldn't EPSG:900913 be replaced by EPSG:3857 as it's now an official EPSG entry? 2) OnEarth Tiled WMS Clementine, WMS srtm and Landsat mosaic do not load $ gdalinfo frmt_twms_Clementine.xml ERROR 1: GDAL_WMS TiledWMS: Can't locate TiledGroup Clementine in server response. ERROR 1: GDALWMS: Failed to initialize minidriver. gdalinfo failed - unable to open 'frmt_twms_Clementine.xml'. Clementine is not found, but the following are returned: NameClementine LO Hybrid, LMMP/Name NameClementine Mosaic, LMMP/Name NameClementine Color Ratio, LMMP/Name NameClementine Optical Maturity, LMMP/Name NameClementine FeO abundance, LMMP/Name NameClementine TiO2 abundance, LMMP/Name $ gdalinfo --debug ON frmt_twms_srtm.xml HTTP: Fetch(http://onearth.jpl.nasa.gov/wms.cgi?request=GetTileService) never ends $ gdal_translate -of JPEG -outsize 500 250 frmt_wms_onearth_global_mosaic.xml onearth_global_mosaic.jpg Input file size is 2949120, 1474560 0 never ends 3) MetaCarta TMS and wmsc - lots of GDAL errors (but gdalinfo is ok), rendering is bad in QGIS 2.2 $ gdal_translate -of PNG -outsize 512 256 frmt_wms_metacarta_tms.xml metacarta.png Input file size is 268435456, 134217728 bunch of errors $ gdal_translate -of PNG -outsize 500 250 frmt_wms_metacarta_wmsc.xml metacarta_wmsc.png Input file size is 268435456, 134217728 bunch of errors 4) TileService, Blue Marble NG - does not load $ gdal_translate -of JPEG -outsize 500 250 frmt_wms_tileservice_bmng.xml tileservice_bmng.jpg Input file size is 65536, 32768 0ERROR 1: GDALWMS: Unable to download block 0, 0. URL: http://s0.tileservice.worldwindcentral.com/getTile?interface=mapversion=1dataset=bmng.topo.bathy.200401level=0x=0y=0 HTTP status code: 404, error: (null). ERROR 1: frmt_wms_tileservice_bmng.xml, band 1: IReadBlock failed at X offset 0, Y offset 0 5) TileService, NYSDOP 2004 - does not load $ gdal_translate -of JPEG -projwin -73.687030 41.262680 -73.686359 41.262345 -outsize 500 250 frmt_wms_tileservice_nysdop2004.xml tileservice_nysdop2004.jpg Input file size is 1048576, 1048576 Computed -srcwin 629496 165307 500 250 from projected window. 0ERROR 1: GDALWMS: Unable to download block 1229, 322. URL: http://s0.tileservice.worldwindcentral.com/getTile?interface=mapversion=1dataset=us.newyork_nysdop2004_0.15mlevel=19x=154829y=70978 HTTP status code: 404, error: (null). ERROR 1: frmt_wms_tileservice_nysdop2004.xml, band 1: IReadBlock failed at X offset 1229, Y offset 322 ERROR 1: GetBlockRef failed at X block offset 1229, Y block offset 322 cheers Etienne ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] Misc. subjects : OSGeo Vienna code sprint, release plans, GDAL 2.0
On Tue, Apr 1, 2014 at 5:57 PM, Kyle Shannon k...@pobox.com wrote: On Tue, Apr 1, 2014 at 12:36 PM, Etienne Tourigny etourigny@gmail.com wrote: The 2 objections I have with json are : - it is so verbose that editing by hand is not as easy as .csv - the xml tags make file size much larger than .csv files, unless they would be stored in a compressed file (gzip) On the other hand, who messes with theses files on a regular basis anyway? It seems like a nice idea. In what ways would it be better than Even's suggestion to use sqlite? On Mon, Mar 31, 2014 at 4:37 PM, Dmitriy Baryshnikov bishop@gmail.com wrote: Hi, I think the JSON format is good for metadata storing and representation. JSON support to store string, digits, bool, dates, etc. Write such data and read such data from files. But need good library or set of classes to work with it in memory representation. For example I like wxJSON - it support to get value by name or iterate values by index. int majorVer = root[wxWidgets][Version][Major].AsInt(); root[wxWidgets][Authors][0] = J. Smart; root[wxWidgets][Authors][1] = V. Zeitling; root[wxWidgets][Authors][2] = R. Roebling; It consist only 3 files (json reader, json writer and json value). It seems to me that json-c library is more complicated. Unfortunately wxJSON cannot be used in GDAL as it have bindings to wxWidgets library, but the approach is interesting. By the way wxWidgets have more democratic license instead of qt. Best regards, Dmitry 31.03.2014 23:03, Even Rouault пишет: Hi Etienne, Thanks for your ideas. Hi all, I have a few suggestions for gdal 2.0, based on my personal experience in learning to use, enhance and maintain gdal/ogr code. - replace cpl/csl/string/xml code with a mainstream, modern cross-platform toolkit such as QT, boost, etc. QT is certainly a dependency we wouldn't want to draw. Too big for some embededded usage, and it would make GDAL to be practially bound by the LGPL. I guess standard C++ libraries classes, or perhaps boost, should do the job for what you mention below. While cpl/csl classes are robust and do the job, they are not well documented and not very intuitive for a new gdal coder. This is from my personal experience, some may not agree. They are also not used outside gdal, as such do not benefit from enhancements as other toolkits. Well, at least, MapServer uses a few CPL functions : CPL minixml, CPLMalloc/CPLFree, CPLReadDir, CPLFormFilename, CPLGetFilename, CSLInsertString, etc.. The drawback is that some data/methods would not be easily available in c and other bindings. Also it might not be available for all platforms. Existing code would still be able to use cpl/csl code but be deprecated until a given release. - upgrade/migrate gdal support file (files in .csv format such as gcs.csv) reading to use containers (e.g. hash maps) instead of reading the relevant .csv files every time Current reading of gdal support files is sub-optimal as any query for support data requires reading the relevant support file(s). It would be more efficient to read these once and cache into a container (such as hash map). This could be done using a cross-platform toolkit mentioned earlier. Another option would be to use a .sqlite database with proper indices. SQLite would become a runtime requirement, but I don't think that would be a big problem. - modify metadata treatment to be able to store/retrieve data in formats other than strings (e.g. floats, doubles, boolean), and query the actual type of a given metadata item. Currently there is no way to know if a given metadata item represents an integer, float, double (or string). In the netcdf driver, I overcame this limitation in 2 ways: 1) parsing the value to test if it is an int, float or double (potentially error-prone) 2) add an extra metadata item to specify the type of the data represented (cumbersome and non-standard) My suggestion would be to be able to store metadata value along with its data type (default string as previously). This would require some kind of container for internal representation (e.g. in qt: QMapQString, QVariant) and a way to represent this on file (as .aux.xml or otherwise in files that support metadata) such as gtiff. Values could be stored as strings to avoid big/little endian conversion problems. Data type could be stored in a (hidden) metadata domain, one item for each real metadata key. Interesting idea. Well, we need coders now ;-) Even ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org
Re: [gdal-dev] alpha mask to data value
On Wed, Apr 2, 2014 at 8:10 PM, William Kyngesburye wokl...@kyngchaos.comwrote: On Apr 2, 2014, at 4:33 PM, Even Rouault even.roua...@mines-paris.org wrote: Le mardi 01 avril 2014 22:14:11, William Kyngesburye a écrit : I've wrestled with various nodata issues in the past, now it's hitting me again... I'm using Photoshop to delete collars on scanned maps, creating an alpha mask. GDAL has no problem with this. What I want to do is merge maps together (after rectification), then set any remaining nodata areas to white, RGB 255,255,255. Just dropping the extra alpha band doesn't work because nodata is set to 0,0,0, which is black. The a_nodata option in gdal_translate just defines what existing value in the data is nodata. I tried using gdalwarp with the -dstnodata option which should set nodata values in the output to a specific value, but it carries along the alpha band and ignores dstnodata. gdalwarp -dstnodata 255 255 255 in.tif out.tif Processing input file in.tif. Using band 4 of source image as alpha. Using band 4 of destination image as alpha. You can perhaps try adding -wo INIT_DEST=255,255,255,0 That appears to work, though not immediately. The raster has 255 in the nodata areas, but it's not until I strip out the alpha band with gdal_translate that it shows. Now I see an odd artifact, or maybe bug somewhere - with nodata values in an RGB image, if any one or two of the bands for a pixel are 255 while the other bands are not, the pixel as a whole is treated as nodata when displaying in QGIS. you may wish to file a bug report in the QGIS tracker (if there is none yet). Though identifying such a pixel will show nodata for just the band(s) that are 255. Photoshop has no concept of nodata values, so it displays correctly (and with white for fully nodata pixels). I think this explains another issue I saw at one point when I had ovelap when merging 2 rasters: the pixels of the top raster covered the pixels of the lower at the band level, which caused the band of the lower to show through when the upper band was 255, and resulted in a wild color mix. ie: result 127 127 127 (grey) but should have been 255 127 127 ^ upper 255 127 127 (pink) ^ lower 127 127 127 (grey) I think I'll save the nodata flattening to white until the end, after merging. - William Kyngesburye kyngchaos*at*kyngchaos*dot*com http://www.kyngchaos.com/ Earth: Mostly harmless - revised entry in the HitchHiker's Guide to the Galaxy ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] Misc. subjects : OSGeo Vienna code sprint, release plans, GDAL 2.0
On Mon, Mar 31, 2014 at 4:03 PM, Even Rouault even.roua...@mines-paris.orgwrote: Hi Etienne, Thanks for your ideas. Hi all, I have a few suggestions for gdal 2.0, based on my personal experience in learning to use, enhance and maintain gdal/ogr code. - replace cpl/csl/string/xml code with a mainstream, modern cross-platform toolkit such as QT, boost, etc. QT is certainly a dependency we wouldn't want to draw. Too big for some embededded usage, and it would make GDAL to be practially bound by the LGPL. I guess standard C++ libraries classes, or perhaps boost, should do the job for what you mention below. In light of this QT is not a candidate, but on the other hand standard c++ library classes only deal with containers right? Are there any cross-platfform, convenience classes? I'm thinking along the lines of things like file paths, xml and csv file reading/writing support. While cpl/csl classes are robust and do the job, they are not well documented and not very intuitive for a new gdal coder. This is from my personal experience, some may not agree. They are also not used outside gdal, as such do not benefit from enhancements as other toolkits. Well, at least, MapServer uses a few CPL functions : CPL minixml, CPLMalloc/CPLFree, CPLReadDir, CPLFormFilename, CPLGetFilename, CSLInsertString, etc.. I don't know much about MapServer, but it seems that GDAL is an optional dependency. Could the CPL code be decoupled from GDAL as an external? Would it make sense or just too much a hassle? The drawback is that some data/methods would not be easily available in c and other bindings. Also it might not be available for all platforms. Existing code would still be able to use cpl/csl code but be deprecated until a given release. - upgrade/migrate gdal support file (files in .csv format such as gcs.csv) reading to use containers (e.g. hash maps) instead of reading the relevant .csv files every time Current reading of gdal support files is sub-optimal as any query for support data requires reading the relevant support file(s). It would be more efficient to read these once and cache into a container (such as hash map). This could be done using a cross-platform toolkit mentioned earlier. Another option would be to use a .sqlite database with proper indices. SQLite would become a runtime requirement, but I don't think that would be a big problem. .sqlite wouls be more compact, but not as easy to edit. - modify metadata treatment to be able to store/retrieve data in formats other than strings (e.g. floats, doubles, boolean), and query the actual type of a given metadata item. Currently there is no way to know if a given metadata item represents an integer, float, double (or string). In the netcdf driver, I overcame this limitation in 2 ways: 1) parsing the value to test if it is an int, float or double (potentially error-prone) 2) add an extra metadata item to specify the type of the data represented (cumbersome and non-standard) My suggestion would be to be able to store metadata value along with its data type (default string as previously). This would require some kind of container for internal representation (e.g. in qt: QMapQString, QVariant) and a way to represent this on file (as .aux.xml or otherwise in files that support metadata) such as gtiff. Values could be stored as strings to avoid big/little endian conversion problems. Data type could be stored in a (hidden) metadata domain, one item for each real metadata key. Interesting idea. Well, we need coders now ;-) Even -- Geospatial professional services http://even.rouault.free.fr/services.html ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] Misc. subjects : OSGeo Vienna code sprint, release plans, GDAL 2.0
The 2 objections I have with json are : - it is so verbose that editing by hand is not as easy as .csv - the xml tags make file size much larger than .csv files, unless they would be stored in a compressed file (gzip) On the other hand, who messes with theses files on a regular basis anyway? It seems like a nice idea. In what ways would it be better than Even's suggestion to use sqlite? On Mon, Mar 31, 2014 at 4:37 PM, Dmitriy Baryshnikov bishop@gmail.comwrote: Hi, I think the JSON format is good for metadata storing and representation. JSON support to store string, digits, bool, dates, etc. Write such data and read such data from files. But need good library or set of classes to work with it in memory representation. For example I like wxJSON - it support to get value by name or iterate values by index. int majorVer = root[wxWidgets][Version][Major].AsInt http://wxcode.sourceforge.net/docs/wxjson/classwxJSONValue.html#9223209e7eea4dd4d668869b69560acb(); root[wxWidgets][Authors][0] = J. Smart; root[wxWidgets][Authors][1] = V. Zeitling; root[wxWidgets][Authors][2] = R. Roebling; It consist only 3 files (json reader, json writer and json value). It seems to me that json-c library is more complicated. Unfortunately wxJSON cannot be used in GDAL as it have bindings to wxWidgets library, but the approach is interesting. By the way wxWidgets have more democratic license instead of qt. Best regards, Dmitry 31.03.2014 23:03, Even Rouault пишет: Hi Etienne, Thanks for your ideas. Hi all, I have a few suggestions for gdal 2.0, based on my personal experience in learning to use, enhance and maintain gdal/ogr code. - replace cpl/csl/string/xml code with a mainstream, modern cross-platform toolkit such as QT, boost, etc. QT is certainly a dependency we wouldn't want to draw. Too big for some embededded usage, and it would make GDAL to be practially bound by the LGPL. I guess standard C++ libraries classes, or perhaps boost, should do the job for what you mention below. While cpl/csl classes are robust and do the job, they are not well documented and not very intuitive for a new gdal coder. This is from my personal experience, some may not agree. They are also not used outside gdal, as such do not benefit from enhancements as other toolkits. Well, at least, MapServer uses a few CPL functions : CPL minixml, CPLMalloc/CPLFree, CPLReadDir, CPLFormFilename, CPLGetFilename, CSLInsertString, etc.. The drawback is that some data/methods would not be easily available in c and other bindings. Also it might not be available for all platforms. Existing code would still be able to use cpl/csl code but be deprecated until a given release. - upgrade/migrate gdal support file (files in .csv format such as gcs.csv) reading to use containers (e.g. hash maps) instead of reading the relevant .csv files every time Current reading of gdal support files is sub-optimal as any query for support data requires reading the relevant support file(s). It would be more efficient to read these once and cache into a container (such as hash map). This could be done using a cross-platform toolkit mentioned earlier. Another option would be to use a .sqlite database with proper indices. SQLite would become a runtime requirement, but I don't think that would be a big problem. - modify metadata treatment to be able to store/retrieve data in formats other than strings (e.g. floats, doubles, boolean), and query the actual type of a given metadata item. Currently there is no way to know if a given metadata item represents an integer, float, double (or string). In the netcdf driver, I overcame this limitation in 2 ways: 1) parsing the value to test if it is an int, float or double (potentially error-prone) 2) add an extra metadata item to specify the type of the data represented (cumbersome and non-standard) My suggestion would be to be able to store metadata value along with its data type (default string as previously). This would require some kind of container for internal representation (e.g. in qt: QMapQString, QVariant) and a way to represent this on file (as .aux.xml or otherwise in files that support metadata) such as gtiff. Values could be stored as strings to avoid big/little endian conversion problems. Data type could be stored in a (hidden) metadata domain, one item for each real metadata key. Interesting idea. Well, we need coders now ;-) Even ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] Misc. subjects : OSGeo Vienna code sprint, release plans, GDAL 2.0
Hi all, I have a few suggestions for gdal 2.0, based on my personal experience in learning to use, enhance and maintain gdal/ogr code. - replace cpl/csl/string/xml code with a mainstream, modern cross-platform toolkit such as QT, boost, etc. While cpl/csl classes are robust and do the job, they are not well documented and not very intuitive for a new gdal coder. This is from my personal experience, some may not agree. They are also not used outside gdal, as such do not benefit from enhancements as other toolkits. The drawback is that some data/methods would not be easily available in c and other bindings. Also it might not be available for all platforms. Existing code would still be able to use cpl/csl code but be deprecated until a given release. - upgrade/migrate gdal support file (files in .csv format such as gcs.csv) reading to use containers (e.g. hash maps) instead of reading the relevant .csv files every time Current reading of gdal support files is sub-optimal as any query for support data requires reading the relevant support file(s). It would be more efficient to read these once and cache into a container (such as hash map). This could be done using a cross-platform toolkit mentioned earlier. - modify metadata treatment to be able to store/retrieve data in formats other than strings (e.g. floats, doubles, boolean), and query the actual type of a given metadata item. Currently there is no way to know if a given metadata item represents an integer, float, double (or string). In the netcdf driver, I overcame this limitation in 2 ways: 1) parsing the value to test if it is an int, float or double (potentially error-prone) 2) add an extra metadata item to specify the type of the data represented (cumbersome and non-standard) My suggestion would be to be able to store metadata value along with its data type (default string as previously). This would require some kind of container for internal representation (e.g. in qt: QMapQString, QVariant) and a way to represent this on file (as .aux.xml or otherwise in files that support metadata) such as gtiff. Values could be stored as strings to avoid big/little endian conversion problems. Data type could be stored in a (hidden) metadata domain, one item for each real metadata key. cheers, Etienne On Thu, Feb 13, 2014 at 7:14 PM, Even Rouault even.roua...@mines-paris.orgwrote: Hi, I've confirmed my presence to the OSGeo Vienna code sprint ( http://wiki.osgeo.org/wiki/Vienna_Code_Sprint_2014 ). Are there folks that will be there and indent doing some work on GDAL ? Any particular topics of interest ? It could be the opportunity to take a crack at some changes that have been mentionned for some time and are listed at http://trac.osgeo.org/gdal/wiki/GDAL20Changes We should decide how to manage the 2.0 transition. I think that the 2.0 number should be reserved as the opportunity of introducing breaking changes in the API / ABI. But this might take a long time to implement all that is listed above, so there's a risk we end up with a trunk that is never ready for release for years. Not a good thing. On the other hand, we could just be more modest and take breaking changes as they are introduced and raise the major version number (so the yearly version after 2.0 could be 3.0). This will require projects using GDAL to adapt multiple times. An alternative would be to prepare the API for new features even if they are not implemented, but that's a difficult exercise and there's a risk that at implementation time, the API doesn't fit. Any thoughts ? Currently we have no such breakage in trunk so it could qualify as GDAL 1.11. Perhaps we should just release it as such for now before the bigger changes ? Somes topics I can see for GDAL 2.0 that impact API/ABI : - well, the mythological unification of OGR driver model with GDAL driver model. - XYZM support - Curve geometries - 64 bit integer support Other possible structural changes : - Change of master version control system: switch to git / GitHub ? - New build system : cmake ? Of course all of this will more likely happen if contributors or funders show up ! Even -- Geospatial professional services http://even.rouault.free.fr/services.html ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] Syncing two rasters together...
I've just heard of this project - any reason (aside from licencing issues) that it could not be added to gdal/ogr itself? It would be cool for gdal 2.0... cheers Etienne On Fri, Mar 28, 2014 at 6:49 AM, kempenep kempe...@gmail.com wrote: Hi Jonathan, If I understand your question correctly, I might suggest an open source tool (GPL v3) for this job: pkcrop. It is part of pktools (http://pktools.nongnu.org), which is based on the GDAL API. By combining pkcrop with pkinfo (also in pktools), you could get what you want in a quite operational way. pkinfo allows you to define the proper bounding box, x, y resolution (pkinfo -i reference.tif -bb -dx -dy). The output can be directly used within pkcrop using $(command substitution). Not covered areas will have a nodata value. You can perform all steps in a one-liner: stacking a list of input images to a multi-layer image using the same bounding box and spatial resolution pkcrop $(pkinfo -i reference.tif -dx -dy -bb) $(for LS in landsat*.tif; do echo -i $LS;done) -o stacked.tif Pieter. -- View this message in context: http://osgeo-org.1560.x6.nabble.com/gdal-dev-Syncing-two-rasters-together-tp5131677p5131753.html Sent from the GDAL - Dev mailing list archive at Nabble.com. ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] Syncing two rasters together...
I admit I did not look too much into your application.
In fact, it would be great to add a reference to it (and other software
that uses GDAL) somewhere in the wiki, perhaps the FAQ page?
Perhaps a few of the tools could be added to the utilities or the
functionality be integrated into existing utilities?
I am not sure about the licensing... pktools is licensed under GPLv3 but
GDAL itself is licensed under the a X11/MIT licence which is NOT GPL.
So I am not sure if some parts could be integrated into GDAL without
changing the licence or corrupting (nothing negative here) GDAL itself.
Many applications use GDAL/OGR because of its more permissive licence. Any
GPL3 code has to be re-licensed (LGPL seems ok) before inclusion into the
gdal source tree.
cheers
Etienne
On Fri, Mar 28, 2014 at 1:13 PM, kempenep kempe...@gmail.com wrote:
Hi,
pktools is licensed under GPL v3. To my understanding, code from pktools
can
therefore be used GDAL. I'm not sure what this means for the resulting
code?
I'm sure we can work it out. All code is in C++, which should facilitate
this (templates are used though).
I am happy to contribute to GDAL if there is interest in selecting
functionality of pktools. In an attempt not to overlap too much with the
GDAL utilities, pktools focuses on remote sensing processing (e.g., image
compositing, support vector machine classification,...). The two utilities
I
mentioned in my previous reply are a bit of an exception, as they overlap
with gdalinfo (-pkinfo) and {gdal_translate, gdal_merge.py, gdalwarp}
(-pkcrop). The main reason for duplicating is this piping functionality
in the example. The output of pkinfo can directly be used in other
utilities
such as pkcrop. There is no need to parse the output from gdalinfo first.
Pieter.
--
View this message in context:
http://osgeo-org.1560.x6.nabble.com/gdal-dev-Syncing-two-rasters-together-tp5131677p5131898.html
Sent from the GDAL - Dev mailing list archive at Nabble.com.
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] Draft GDAL/OGR class hierarchy for GDAL 2.0
How about multiple inheritance for mixed class? Say you have a class for rasters and a class for vectors, and a dataset that would support both types would inherit from these two base classes. Is that what you mean by composite? Etienne On Thu, Mar 27, 2014 at 9:32 AM, Mateusz Łoskot mate...@loskot.net wrote: On 27 March 2014 11:24, Even Rouault even.roua...@mines-paris.org wrote: Is it possible to put it as a member in classes ? That could have been a way of doing it, yes. Always the debate composition vs inheritance. The base classes for raster and vector datasets could have common base class. The mixed raster/vector dataset could be defined as a composite of the former two. Best regards, -- Mateusz Łoskot, http://mateusz.loskot.net ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] Open WMS datasets with GDAL Java
Have you tried running in debug mode (CPL_DEBUG=ON)? On Thu, Mar 27, 2014 at 3:37 PM, Fabio Rinnone fabio.rinno...@gmail.comwrote: Il 27/03/2014 18:27, Even Rouault ha scritto: Selon Fabio Rinnone fabio.rinno...@gmail.com: Hi, I'm trying to open datasets of WMS service with GDAL Java warp in an Android app project with native support. Opening datasets always returns null datasets. An example WMS is the following: http://kaart.maakaart.ee/geoserver/wms I use following statements: name = WMS:http://kaart.maakaart.ee/geoserver/wms;; gdal.Open(name); For my implementation I have adapted python sample code found here[2]. I guess you lack a call to gdal.AllRegister(); that will register the WMS driver (in Python the registration of drivers is automatically done) I'm tried but I have no solved the issue. I thought the problem could be related to the inclusion of curl libraries (which are required to support WMS), but they are correctly included in Android.mk makefile. Do you have other suggestions? -- Fabio Rinnone 93015 Niscemi CL, Italia Skype: fabiorinnone Web: http://www.fabiorinnone.eu ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] Draft GDAL/OGR class hierarchy for GDAL 2.0
Dmitry - I am not sure that a wholescale refactor is the way to go - many formats will be dropped because quite a few do not have active maintainers, and development is mostly on a volunteer basis (correct me if I am wrong, Even). As an example, the netcdf driver has been mostly maintained by myself for the past few years (with the help of a few users and devs), and it has been about 99% voluntary work. And I will be unable to work on it for the next year, so it probably would not be upgraded if the API changes drastically. And I am sure there are quite a few drivers in the same situation. It there a way to think of a new design, while retaining backwards compatibility for the drivers that would not be updated? Especially for the raster drivers, which would probably need less changes. cheers Etienne On Thu, Mar 27, 2014 at 4:13 PM, Dmitriy Baryshnikov bishop@gmail.comwrote: Hi Even, I think that we can discuss the starting point for 2.0. You start with idea ...to refactor a code base of 1.2 million lines.. Maybe we have to start from white list - create ideal (brilliant) set of classes and create a well-designed foundation for GDAL. Also rewrite several wide used drivers (shp, tab, PostGIS, etc.). Than this will works very good, begin some kind of hackathon for rewriting overs. Also we can add some testing framework inside GDAL (e.g. googletest) and start empty repo on github for GDAL 2.0. The GDAL can be enhanced, we can move common things that use several drivers to core or replace some core thing with outer libraries. CMake maybe used as build system. Yes this is deep refactoring, but maybe this is need for GDAL 2.0? Best regards, Dmitry 27.03.2014 14:24, Even Rouault пишет: Hi, Hi all, MY opinion is that's not a really good way to count how many classes you will have and trying to have minimal classes. Drawing a model with UML is not easy : it's not easy to have a good model from the beginning, it's not easy to name all members and methods correctly, and the worst is inheritance. You can think you spend too much time to make the model. But you can't say if you will spend less or much time to change code because your model was too simple. The issue is that I'm trying to refactor a code base of 1.2 million lines of C/C++ code (not mentionning the 140 000 lines of Python tests), and with 211 existing drivers. So, given that nobody will probably ever want to fund the effort, it is completely excluded to have to edit each of those drivers in a non automated way... The end solution will necessary be a compromise. I would also like the existing C API to continue working as much as possible. Because I don't know all the model of Gdal and the model can be reset I have some questions : 1) Why GDALMajorObject not inherit of GDALIMajorObject ? It was just an omission in the UML diagram. My C++ sketch had this inheritance. 2) Why all classes inherit (directly or not) of GDALMajorObject ? That's how it is done currently. What do you store in it ? It contains a list of list of strings, or in more concrete terms, metadata domains that contain metadata items. Is it possible to put it as a member in classes ? That could have been a way of doing it, yes. Always the debate composition vs inheritance. Have you consider templates ? Actually, no. I tend to avoid them unless they are an obvious solution. 3) An interface for Drivers ? geName(), getLongName(), getDescription() ? Or maybe abstract ... 4) What is GDALDataset in your diagram ? It is the base class that a driver that supports both raster and vector would have to subclass. 5) Where is the link between Driver (Gdal/Ogr) and Dataset (Raster/Vector) ? Member, templates ? A Driver instanciates a Dataset by opening an existing one, or by creating a new one. But the Dataset object itself is a free-standing objects that is only owned by user code, and not by the Driver. For now, I've dropped my crazy UML diagram that nobody would be able to understand, even if I managed to make it work, and I'm pursuing my attempt just merging OGRDataSource inside GDALDataset. And make OGRDriver and OGRSFDriverRegistrar mostly disappear, or just create a minimized version of them, as compatibility layer for existing drivers, that forward the real work to GDALDriver and GDALDriverManager. Even Florent 2014-03-25 21:14 GMT+01:00 Even Rouault even.roua...@mines-paris.org: Selon Etienne Tourigny etourigny@gmail.com: I also (respectfully) think that there are too many classes (mostly the abstract classes and interfaces) which make it a bit hard to understand. I like Vincent's second suggestion to have 3 base Dataset classes (GDALRasterDataset, GDALVectorDataset, GDALHybridDataset) which inherit from GDALDataset, and an enum (vector,raster,hybrid) to easily query which type the Dataset is (or which type its driver supports). You
Re: [gdal-dev] Draft GDAL/OGR class hierarchy for GDAL 2.0
I also (respectfully) think that there are too many classes (mostly the abstract classes and interfaces) which make it a bit hard to understand. I like Vincent's second suggestion to have 3 base Dataset classes (GDALRasterDataset, GDALVectorDataset, GDALHybridDataset) which inherit from GDALDataset, and an enum (vector,raster,hybrid) to easily query which type the Dataset is (or which type its driver supports). You then subclass the relevant Dataset class and it *should* work auto-magically for rasters. For example, a given Dataset which subclasses GDALDataset would be migrated as a subclass of GDALRasterDataset, which would imply that it does not have a vector dataset. If the implementation changes to allow vector, then you would change its parent and implement relevant vector methods. My view on this is KISS with a minimum number of classes... cheers Etienne On Tue, Mar 25, 2014 at 9:13 AM, Vincent Mora vincent.m...@oslandia.comwrote: On 25/03/2014 11:44, Even Rouault wrote: Selon Vincent Mora vincent.m...@oslandia.com: On 24/03/2014 21:46, Even Rouault wrote: Hi, Release soon, release early, so for people who like UML diagrams (there is also a prototype of C++ classes for those who don't like UML very much), here's a blog entry with the outcome of my thoughts for a possible re-organisation of the GDAL/OGR class hierarchy http://erouault.blogspot.ca/2014/03/draft-gdalogr-class- hierarchy-for-gdal.html I don't understand the need for HandleRasterData() and HandleVectorData(). Isn't inheriting from GDALEmptyRasterDataset or GDALEmptyVectorDataset sufficient to convey the information that the class doesn't handle those kind of datasets. A call to GetLayerCount()/GetRasterCount() from the class user is sufficient to say no vector/raster data here, and a dynamic _cast to GDALEmptyVectorDataset/GDALEmptyRasterDataset is even clearer to me, so HandleRasterData()/HandleVectorData() would be a third way to tell the same thing. Actually, when thinking more about this, this kind of information should be at the driver level, and not at the dataset level. The idea is that there are use cases where you want to know if a driver can handle only vector, only raster, or both. For example if you still want to have separate Open dialog boxes for raster or vector. Also I'm not sure about the names GDALAbstract* since those are partial implementations. I know I'm not very good at naming. Do you have an alternative proposal ? The issue is that with my draft we end up with a lot of classes and interfaces, so it is not obvious to find a good name to reflect their content. Partial sound ok to me, but I not so good at naming ? Considering the number of classes, I gave it some thoughts this morning and, like Dmitry, thought of merging the Abtract (Partial) into the interface before realizing that, even if it works (you can overload de default function in Empty) it's a bit ugly and I ended up preferring your solution. An altenative would be to have a Dataset with both aspects and provide three partial specialisations: one for vector (it will behave like empty raster) one for raster, and one for both. Code duplications could be avoided by implementing protected member functions in the Dataset class and simply calling them in the implementations of partial specialization. This solution avoid the diamond shaped inheritance diagram of hell :) Even ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] How to find the geoextents in a netCDF file
As Even suggests you should read the tutorial first, most questions will be answered there. As for Band extraction, you need to look at the metadata, which you can inspect with gdalinfo you should look for metadata like NETCDF_DIM_time_DEF NETCDF_DIM_time_VALUES and band metadata like NETCDF_DIM_time=1 If you want more fine-grained control on band selection based on a particular dimension (in this case time), then perhaps you should look into software that specializes in netcdf format like nco or cdo. On Tue, Mar 18, 2014 at 7:07 AM, sandeep sandeepkumar7...@gmail.com wrote: hi, i had run gdalinfo on .nc file. it doesnt have subdatsets. it is rainfall data. it has 122 bands. how can i get geoextents of this file. and how do i extract each band from it if i want display band according to the time? how can i access those variables? i am writing this in C so i want to know how to access the variables in this file? thanks sandeep -- View this message in context: http://osgeo-org.1560.x6.nabble.com/How-to-find-the-geoextents-in-a-netCDF-file-tp5129619.html Sent from the GDAL - Dev mailing list archive at Nabble.com. ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] How to find the geoextents in a netCDF file
the gdal api tutorial will give you pointers on how to access dataset and band-level metadata On Tue, Mar 18, 2014 at 2:38 PM, sandeep sandeepkumar7...@gmail.com wrote: yes Etienne i need to extract band according to the NETCDF_DIM_time_VALUES. so i am doing it in C. so i need to access these variable.. -- View this message in context: http://osgeo-org.1560.x6.nabble.com/How-to-find-the-geoextents-in-a-netCDF-file-tp5129619p5129724.html Sent from the GDAL - Dev mailing list archive at Nabble.com. ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] How to find the geoextents in a netCDF file
loop for each raster as explained in the tutorial, and get the specific metadataitem like this const char* md = GDALGetMetadataItem(hBand, NETCDF_DIM_time_VALUE, ) md will contain the value of NETCDF_DIM_time_VALUE for that band On Tue, Mar 18, 2014 at 2:42 PM, sandeep sandeepkumar7...@gmail.com wrote: From the GetRasterCount() i can find the number of bands and using vrt i am creating a file for every band. in order to to show them according to the NETCDF_DIM_time_VALUES how can i do that? -- View this message in context: http://osgeo-org.1560.x6.nabble.com/How-to-find-the-geoextents-in-a-netCDF-file-tp5129619p5129725.html Sent from the GDAL - Dev mailing list archive at Nabble.com. ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] Fix for QGIS 2.2+ Projection issue with OSGB36 NTv2
You email should be sent to the qgis-developer list (this is the gdal-dev list), which I am forwarding this to. On Tue, Mar 11, 2014 at 7:44 AM, Nicholas Duggan dragons8my...@live.co.ukwrote: Hi all, I'm trying to get my fix for the OSGB36 to WGS84 using Ordnance Surveys OSTN02 (NTv2) transformation into the QGIS master. Nathan Woodrow recommended I bring it up with you guys who might be able to implement it. Essentially I've updated the QGIS srs.db and put the OSTN02_NTv2.gsb in the Share/Proj folder, full description and files are here: http://dragons8mycat.wordpress.com/2014/03/03/gis-tips-getting-ostn02-to-work-with-qgis-2-2/ Is this something which can be implemented by yourselves or can you point me in the right direction for getting it put in the master/future releases? Thanks, Nick D (@dragons8mycat) ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] Extra Decimal places added to ASCII gdal_translate mosaic
Have you tried using a 36bit (float) instead of 64bit (double) output? double is usually the default output type and has more significant digits (~16) than your source, so it's representation is ok in double format see [1] for an explanation on floating-point precision the following argument would do this -ot Float32 [1] http://en.wikipedia.org/wiki/Floating_point Etienne On Mon, Mar 10, 2014 at 2:25 PM, Jonathan Moules jonathanmou...@warwickshire.gov.uk wrote: Hi List, Bringing back an old thread but with a slight change. I'm now trying to get GDAL to mosaic and export from the ASCII files to a GeoTIFF. But the GeoTIFF writer in GDAL is doing the exact same thing as the ASCII writer was - it's taking *50.597* (source value) and converting it into something like *50.596895342423*. I've tried using both gdal_merge and going via a VRT (gdalbuildvrt - gdal_translate). The result is the same. I also tried the *DECIMAL_PRECISION* trick, but gdal informs me: Warning 6: Driver GTiff does not support DECIMAL_PRECISION creation option How do I get GDAL to merge these into a GeoTIFF without altering the data? Thanks, Jonathan On 24 January 2014 16:12, Jonathan Moules jonathanmou...@warwickshire.gov.uk wrote: Thanks Norman. I'd already seen that page but failed to read it! However, having tested it, it's not functioning as the manual says. As best I can tell, DECIMAL_PRECISION=3 should give me 3 decimal places. But what it instead does it give me 3 significant figures. So 98.354 has become 98.3. This becomes a problem because many of my heights are hovering around 100. So some will either have to be 1 character too long to ensure no data loss for the rest. Is this a bug? The documentation on the page linked to is quite clear it should be decimal places. Less of an issue, but it also didn't make any difference to: xllcorner432000. yllcorner242000. cellsize 2. GDAL 1.10.1, released 2013/08/26 Cheers, Jonathan On 24 January 2014 15:11, Norman Vine n...@cape.com wrote: try gdal_translate -of AAIGrid -co DECIMAL_PRECISION=3 abc.vrt abc.asc see http://www.gdal.org/frmt_various.html HTH Norman On Jan 24, 2014, at 10:01 AM, Jonathan Moules jonathanmou...@warwickshire.gov.uk wrote: Hi List, I'm trying to mosaic some ASCII grid tiles into a single large ASCII file (which can then easily be handled). I'm using this process: REM Create list of files dir /b /s *.asc asc_list.txt REM Turn into VRT gdalbuildvrt -srcnodata - -vrtnodata 0 -a_srs EPSG:27700 -input_file_list asc_list.txt abc.vrt REM actual mosaicing gdal_translate -of AAIGrid abc.vrt abc.asc The input files are like: ncols 500 nrows 500 xllcorner 432000.000 yllcorner 243000.000 cellsize 2 nodata_value -.0 98.354 98.449 98.658 98.874 99.038 99.096 But the outputs are like: fncols500 nrows1000 xllcorner432000. yllcorner242000. cellsize 2. NODATA_value 0 98.353996276855469 98.448997497558594 98.657997131347656 98.874000549316406 Suddenly everything has a dozen extra decimal places. Not only does this induce the False Precision problem (we don't have heights to the femtometer!), it also massively increases the filesize. How can I get this to not happen? Thanks, Jonathan This transmission is intended for the named addressee(s) only and may contain sensitive or protectively marked material up to RESTRICTED and should be handled accordingly. Unless you are the named addressee (or authorised to receive it for the addressee) you may not copy or use it, or disclose it to anyone else. If you have received this transmission in error please notify the sender immediately. All email traffic sent to or from us, including without limitation all GCSX traffic, may be subject to recording and/or monitoring in accordance with relevant legislation. ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev This transmission is intended for the named addressee(s) only and may contain sensitive or protectively marked material up to RESTRICTED and should be handled accordingly. Unless you are the named addressee (or authorised to receive it for the addressee) you may not copy or use it, or disclose it to anyone else. If you have received this transmission in error please notify the sender immediately. All email traffic sent to or from us, including without limitation all GCSX traffic, may be subject to recording and/or monitoring in accordance with relevant legislation. ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev ___ gdal-dev mailing list
Re: [gdal-dev] Moving the poles
Not sure about gdal, but cdo [1] can do this on netcdf files. have a look at the docs and see the section grids. You would probably have to define a rotated pole grid, then apply that grid on your file (using setgrid), then regrid to a normal lat/lon grid using one of the grid operators like interpolate, remapnn, remapbil, depending on your data. But I've never done this though... There us a forum there if you have questions, someone probably has an answer. [1] https://code.zmaw.de/projects/*cdo* On Sun, Mar 9, 2014 at 12:29 AM, Laskaris francisray...@yahoo.co.uk wrote: I am trying to do a crazy What if...? climate model experiment that involves changing the location of the poles. So, mind you, I am not merely trying to change the coordinate display and use a rotated pole grid. I am actually trying to locate the pole somewhere else (for instance, in Australia), and transform all the elevation etc. data accordingly to new coordinates. Is it possible to transform a geotiff or netcdf file with GDAL to achieve this? -- View this message in context: http://osgeo-org.1560.x6.nabble.com/Moving-the-poles-tp5127897.html Sent from the GDAL - Dev mailing list archive at Nabble.com. ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] Problem with results on two different versions of gdal_wrap
Perhaps differences in the hdf driver between the two versions could explain differences in output. Try converting netcdf file to gtiff with gdal_translate (using one gdal version), then try gdalwarp from that common gtiff file. On Sat, Mar 8, 2014 at 7:22 AM, Even Rouault even.roua...@mines-paris.orgwrote: This is the problem step: gdalwarp -rcs -ts 8800 6600 -s_srs EPSG:32662 -t_srs EPSG:4326 temp.tif target.tif gdalinfo -mm -stats target.tif is showing that the range of values in the image are dramatically different on the two servers! summary old: Band 1 Block=8800x1 Type=Int16, ColorInterp=Gray Computed Min/Max=-3877.000,32767.000 Minimum=-3877.000, Maximum=32767.000, Mean=25235.731, StdDev=10612.642 summary new: Band 1 Block=8800x1 Type=Int16, ColorInterp=Gray Min=-9314.000 Max=32561.000 Computed Min/Max=-9314.000,32561.000 Minimum=-9314.000, Maximum=32561.000, Mean=19166.800, StdDev=7786.806 Ok, so you can see that the values are radically different. My question is how do I get values like the old system? These values represent temperatures and I need to get the same values. My one thought on this is that if is another side effrct of proj4 behaving differently as I had to adjust the position above to get it to align. So maybe the gdalwarp is messing up the pixel values when it reprojects it also. But I'm totally lost on how make this work correctly. Any thoughts on how to fix this? Stephen, I think we already had a discussion some time ago about differences between spherical or ellipsoidal projections, or am I confusing with someone else ? Well, it is not clear from your experiment if the difference is due to reprojection or to the resampling method. There's a difference between both GDAL versions, but is the new result worse than the previous one (from visual inspection) ? Cubic spline resampling seems to produce overshoot artifacts in both situations (since -3877.000 or -9314.000 in output 377 in input). That's probably due to the maths behind. Maybe just try with the default nearest resampling to see if it is due to the resampling kernel or the reprojection. I'm also wondering if your data doesn't have a nodata value that you should explicitely set. As I can only guess, 32767 would be a good candidate given that the data type is Int16. But the _FillValue=[65535] in the metadata makes me wonder if the datatype shouldn't be UInt16 rather than Int16 in your initial conversion from netcdf to geotiff, and the nodata would rather be 65535 ? Even -- Geospatial professional services http://even.rouault.free.fr/services.html ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] Something wrong with writing a big raster into PDF
Jukka - have you tried with other pdf viewers? On Fri, Mar 7, 2014 at 1:43 PM, Pepijn Van Eeckhoudt pep...@vaneeckhoudt.net wrote: On 07-03-14 11:42, Jukka Rahkonen wrote: I had another try with the same binaries with a small png file (2000x2000 pixels) and conversion into pdf was fast and successful. Is there perhaps something that does not scale up properly when the raster size gets bigger? Based on my own experience, Adobe's PDF viewer doesn't handles this type of PDF very well. They eventually get displayed but it can take a long time. Internally raster PDFs are typically structured as a large, single level mosaic of small images (jpeg or jpeg2000). There's some fairly large examples at http://www.agc.army.mil/Missions/Terrain/GeoPDFMaps/ GeoPDFGallery.aspx that you could have a look at and see how Acrobat copes with those. Pepijn ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] reprojecting coastcolour (/meris) using python via GCPs
On Tue, Feb 18, 2014 at 9:22 AM, Ivan Price ivan.pr...@noveltis.fr wrote: Hi again Etienne, sorry for being vague.. when i say it 'works' in inverted commas, its because i'm not sure to what level it is expected to work. I can read the list of bands but there is no georeferencing for example., gdal doesn't see the gcps (=values in lat/lon variables). so previously i was writing out a (subsetted) gtiff with gcps in python then warping using gdalwarp -tps which was working well actually.. just a few more steps than i'd ideally like. following your advice regarding the vrt and after learning from nansat examples (thanks Anton!) i now write a vrt with gcps and use gdalwarp from the commandline with -tps on that.. which works fine.. although i need to use a gcp for every 200 pixels or the calculation time is too high. if you tell me that the gcps should be recognised automatically by gdal then i'd be happy to file a bug and upload some sample data. for info i'm using gdal 1.9.2, built by me using flags: --prefix --with-python --pymoddir=xxx and thats all. You might have better success using a more recent gdal version, the netcdf driver has changed a bit since 1.9. I am not sure how it would handle gcps though, I added support for GEOLOCATION arrays but that is a different thing. Seems to me using a vrt is the best option. Etienne thanks for your assistance, -i save.nc is an untouched coastcolour dataset. (price@nemo:)$ gdalinfo NETCDF:save.nc:reflec_1 Driver: netCDF/Network Common Data Format Files: none associated Size is 4481, 9921 Coordinate System is `' Metadata: NC_GLOBAL#auto_grouping=reflec:norm_reflec:atm_tau NC_GLOBAL#Conventions=CF-1.4 NC_GLOBAL#metadata_profile=beam NC_GLOBAL#metadata_version=0.5 NC_GLOBAL#product_type=MER_FSG_CCL2R NC_GLOBAL#start_date=08-FEB-2011 10:37:07.984543 NC_GLOBAL#stop_date=08-FEB-2011 10:44:24.434783 NC_GLOBAL#TileSize=16:4481 NC_GLOBAL#title=MERIS CoastColour L2R reflec_1#bandwidth=9.9370003 reflec_1#coordinates=lat lon reflec_1#long_name=Water leaving irradiance reflectance at 412.691 nm reflec_1#solar_flux=1764.0343 reflec_1#spectral_band_index=0 reflec_1#units=sr^-1 reflec_1#valid_pixel_expression=!l2r_flags.INPUT_INVALID reflec_1#wavelength=412.69101 Corner Coordinates: Upper Left (0.0,0.0) Lower Left (0.0, 9921.0) Upper Right ( 4481.0,0.0) Lower Right ( 4481.0, 9921.0) Center ( 2240.5, 4960.5) Band 1 Block=4481x1 Type=Float32, ColorInterp=Undefined NoData Value=9.96920996838686905e+36 Metadata: bandwidth=9.9370003 coordinates=lat lon long_name=Water leaving irradiance reflectance at 412.691 nm NETCDF_VARNAME=reflec_1 solar_flux=1764.0343 spectral_band_index=0 units=sr^-1 valid_pixel_expression=!l2r_flags.INPUT_INVALID wavelength=412.69101 (price@nemo:)$ gdalwarp -t_srs EPSG:4326 NETCDF:save.nc:reflec_1 foo.tif Processing input file NETCDF:save.nc:reflec_1. ERROR 1: Unable to compute a transformation between pixel/line and georeferenced coordinates for NETCDF:save.nc:reflec_1. There is no affine transformation and no GCPs. *De :* Etienne Tourigny [mailto:etourigny@gmail.com] *Envoyé :* Monday, 17 February 2014 20:32 *À :* Ivan Price *Cc :* gdal-dev@lists.osgeo.org *Objet :* Re: [gdal-dev] reprojecting coastcolour (/meris) using python via GCPs On Mon, Feb 17, 2014 at 2:03 PM, Ivan Price ivan.pr...@noveltis.fr wrote: Hi Etienne, firstly the problem reading the MERIS (actually coastcolour) data was due to the fact i was using gdal 1.6, after upgrading to 1.9 i can at least list the bands so i guess that 'works'. does it really work or sort of? If not, please file a bug report with an example data set attached or a link to it. regarding the use of the filesystem, i tried that, however it is slow to write out the whole dataset to disk, (i need to treat hundreds of images), so i was looking for a way to avoid this.. although it will be my fallback option if all else fails.. have you tried using a .vrt file? either way i'll reply to the list with what we do in the end thanks for your response, -i *De :* Etienne Tourigny [mailto:etourigny@gmail.com] *Envoyé :* Monday, 17 February 2014 14:09 *À :* Ivan Price *Cc :* gdal-dev@lists.osgeo.org *Objet :* Re: [gdal-dev] reprojecting coastcolour (/meris) using python via GCPs As far as I know, the gdal warp api is not exposed to python. I have no idea on reading the MERIS data with the gdal netcdf driver - what I the problem? But you might be able to use this workaround: instead of creating a source dataset using the memory driver, create a file on disk (in gtiff format) with GCPS and write data to disk, and then use gdalwarp on that. Etienne On Mon, Feb 17, 2014 at 6:46 AM
Re: [gdal-dev] reprojecting coastcolour (/meris) using python via GCPs
As far as I know, the gdal warp api is not exposed to python. I have no idea on reading the MERIS data with the gdal netcdf driver - what I the problem? But you might be able to use this workaround: instead of creating a source dataset using the memory driver, create a file on disk (in gtiff format) with GCPS and write data to disk, and then use gdalwarp on that. Etienne On Mon, Feb 17, 2014 at 6:46 AM, Ivan Price ivan.pr...@noveltis.fr wrote: Hello, I am trying to reproject a window inside a coastcolour (=MERIS) image. As far as I can see GDAL cannot read the coastcolour data directly, so i am reading the coastcolour netcdf in python, building a source dataset using the memory driver, adding GCPS (1 for every 10th pixel) and writing the data to it, then reprojecting the source dataset to a destination dataset which is a spatial subset of the original in wgs84 lat/long. This works fine and is relatively fast, but the reprojection is not accurate, the results are out by about 6-10 pixels (in various directions). On reading the forums it seems if i was using gdalwarp i would be using -tps, however the ReprojectImage() function does not seem to offer this parameter ? And i don't have the option of using the commandline tool as even gdal 1.10 cannot recognise the coastcolour data. So i guess i have 2 questions.. has anyone had any success reading coastcolour data with the gdal command line tools, and secondly: how can i get ReprojectImage() to be more accurate, given i have a GCP for every pixel ? thanks and regards, -ivan ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] reprojecting coastcolour (/meris) using python via GCPs
On Mon, Feb 17, 2014 at 2:03 PM, Ivan Price ivan.pr...@noveltis.fr wrote: Hi Etienne, firstly the problem reading the MERIS (actually coastcolour) data was due to the fact i was using gdal 1.6, after upgrading to 1.9 i can at least list the bands so i guess that 'works'. does it really work or sort of? If not, please file a bug report with an example data set attached or a link to it. regarding the use of the filesystem, i tried that, however it is slow to write out the whole dataset to disk, (i need to treat hundreds of images), so i was looking for a way to avoid this.. although it will be my fallback option if all else fails.. have you tried using a .vrt file? either way i'll reply to the list with what we do in the end thanks for your response, -i *De :* Etienne Tourigny [mailto:etourigny@gmail.com] *Envoyé :* Monday, 17 February 2014 14:09 *À :* Ivan Price *Cc :* gdal-dev@lists.osgeo.org *Objet :* Re: [gdal-dev] reprojecting coastcolour (/meris) using python via GCPs As far as I know, the gdal warp api is not exposed to python. I have no idea on reading the MERIS data with the gdal netcdf driver - what I the problem? But you might be able to use this workaround: instead of creating a source dataset using the memory driver, create a file on disk (in gtiff format) with GCPS and write data to disk, and then use gdalwarp on that. Etienne On Mon, Feb 17, 2014 at 6:46 AM, Ivan Price ivan.pr...@noveltis.fr wrote: Hello, I am trying to reproject a window inside a coastcolour (=MERIS) image. As far as I can see GDAL cannot read the coastcolour data directly, so i am reading the coastcolour netcdf in python, building a source dataset using the memory driver, adding GCPS (1 for every 10th pixel) and writing the data to it, then reprojecting the source dataset to a destination dataset which is a spatial subset of the original in wgs84 lat/long. This works fine and is relatively fast, but the reprojection is not accurate, the results are out by about 6-10 pixels (in various directions). On reading the forums it seems if i was using gdalwarp i would be using -tps, however the ReprojectImage() function does not seem to offer this parameter ? And i don't have the option of using the commandline tool as even gdal 1.10 cannot recognise the coastcolour data. So i guess i have 2 questions.. has anyone had any success reading coastcolour data with the gdal command line tools, and secondly: how can i get ReprojectImage() to be more accurate, given i have a GCP for every pixel ? thanks and regards, -ivan ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] Is there documetation about filters and some python utilities?
On Sun, Feb 16, 2014 at 8:36 AM, Jukka Rahkonen jukka.rahko...@mmmtike.fiwrote: Hi, I was thinking that a possibility to run images through filters with GDAL could be nice sometimes. Especially for visual work with aerial and satellite images filters like sharpen of unsharp mask http://en.wikipedia.org/wiki/Unsharp_masking are often very useful. But then I found that kernel filters can already be used inside VRT files. Are there usage examples with before-after comparison images somewhere in our documentation? The only I know of is the vrt tutorial (with very few details), but you have probably already read that http://www.gdal.org/gdal_vrttut.html These refs may help (obtained through a simple google search) http://homepages.inf.ed.ac.uk/rbf/HIPR2/unsharp.htm http://stackoverflow.com/questions/2938162/how-does-an-unsharp-mask-work I also found that there are some python scripts included in \gdal\bin\gdal\python\scripts at least in the gisinternals GDAL builds but some scripts are not listed in http://gdal.org/gdal_utilities.html. Is there some other place where to find documentation about these scripts, for example gdal_calc.py ? Some other than the sources, I mean. -Jukka Rahkonen- ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] warnings a fail with poppler
On Sun, Feb 9, 2014 at 10:21 AM, Even Rouault even.roua...@mines-paris.orgwrote: Le dimanche 02 février 2014 02:05:23, Joaquim Luis a écrit : Hi, I am seeing this warnings and trying to build with poppler (a trunk build here) fails too with pdfdataset.obj : error LNK2001: unresolved external symbol public: __cdecl SplashOutputDev::SplashOutputDev(enum SplashColorMode,int,bool,unsigned char *,bool,bool,enum SplashThinLineMode,bool) (??0SplashOutputDev@@QEAA@W4SplashColorMode @@H_NPEAE11W4SplashThinLineMode@ @1@Z) pdfdataset.obj : error LNK2001: unresolved external symbol public: struct Ref * __cdecl Catalog::getPageRef(int) (?getPageRef@Catalog@@QEAAPEAURef@@H@Z) I could see that SplashOutputDev is poppler.lib I'm not sure why you get this error. Perhaps due how poppler.lib was generated ? Are the symbols well exported in it ? I can see that Tamas has upgraded his builds at http://www.gisinternals.com/sdk/ to latest poppler (0.24.5), so you could likely take poppler.lib from there. and now the warnings They aren't particularly new I guess since that code hasn't changed in a while. Would be nice if someone wanted to clean them up. this kind of warnings appear (or not), depending on the compiler and version used. c:\programs\gdaltrunk\gdal\frmts\iris\irisdataset.cpp(564) : warning C4701: potentially uninitialized local variable 'dfCos2SigmaM' used c:\programs\gdaltrunk\gdal\frmts\iris\irisdataset.cpp(558) : warning C4701: potentially uninitialized local variable 'dfSinSigma' used c:\programs\gdaltrunk\gdal\frmts\iris\irisdataset.cpp(558) : warning C4701: potentially uninitialized local variable 'dfCosSigma' used c:\programs\gdaltrunk\gdal\frmts\hdf4\hdf4imagedataset.cpp(2469) : warning C4701: potentially uninitialized local variable 'iWrkNumType' used c:\programs\gdaltrunk\gdal\frmts\hdf4\hdf4imagedataset.cpp(3440) : warning C4706: assignment within conditional expression c:\programs\gdaltrunk\gdal\frmts\hdf4\hdf4imagedataset.cpp(3447) : warning C4706: assignment within conditional expression c:\programs\gdaltrunk\gdal\frmts\hdf4\hdf4imagedataset.cpp(3462) : warning C4706: assignment within conditional expression c:\programs\gdaltrunk\gdal\frmts\hdf4\hdf4imagedataset.cpp(3469) : warning C4706: assignment within conditional expression c:\programs\gdaltrunk\gdal\frmts\hdf4\hdf4imagedataset.cpp(3401) : warning C4701: potentially uninitialized local variable 'dfNoData' used c:\programs\gdaltrunk\gdal\frmts\hdf4\hdf4imagedataset.cpp(3405) : warning C4701: potentially uninitialized local variable 'dfScale' used c:\programs\gdaltrunk\gdal\frmts\hdf4\hdf4imagedataset.cpp(3410) : warning C4701: potentially uninitialized local variable 'dfOffset' used c:\programs\gdaltrunk\gdal\frmts\hdf4\hdf-eos\ehapi.c(448) : warning C4701: potentially uninitialized local variable 'sdInterfaceID' used c:\programs\gdaltrunk\gdal\frmts\hdf4\hdf-eos\ehapi.c(435) : warning C4701: potentially uninitialized local variable 'HDFfid' used c:\programs\gdaltrunk\gdal\frmts\hdf4\hdf-eos\ehapi.c(445) : warning C4701: potentially uninitialized local variable 'acs' used c:\programs\gdaltrunk\gdal\frmts\hdf4\hdf-eos\gdapi.c(3270) : warning C4701: potentially uninitialized local variable 'c_info' used c:\programs\gdaltrunk\gdal\frmts\hdf4\hdf-eos\gdapi.c(3281) : warning C4701: potentially uninitialized local variable 'chunkDef' used c:\programs\gdaltrunk\gdal\frmts\hdf4\hdf-eos\gdapi.c(6470) : warning C4701: potentially uninitialized local variable 'latrad0' used c:\programs\gdaltrunk\gdal\frmts\hdf4\hdf-eos\gdapi.c(6500) : warning C4701: potentially uninitialized local variable 'yMtr0' used c:\programs\gdaltrunk\gdal\frmts\hdf4\hdf-eos\gdapi.c(6499) : warning C4701: potentially uninitialized local variable 'xMtr0' used c:\programs\gdaltrunk\gdal\frmts\hdf4\hdf-eos\gdapi.c(2632) : warning C4701: potentially uninitialized local variable 'ndims' used c:\programs\gdaltrunk\gdal\frmts\hdf4\hdf-eos\gdapi.c(6741) : warning C4701: potentially uninitialized local variable 'qp_cea' used c:\programs\gdaltrunk\gdal\frmts\hdf4\hdf-eos\gdapi.c(6741) : warning C4701: potentially uninitialized local variable 'kz_cea' used c:\programs\gdaltrunk\gdal\frmts\hdf4\hdf-eos\gdapi.c(7967) : warning C4701: potentially uninitialized local variable 'minCol' used c:\programs\gdaltrunk\gdal\frmts\hdf4\hdf-eos\gdapi.c(7969) : warning C4701: potentially uninitialized local variable 'minRow' used c:\programs\gdaltrunk\gdal\frmts\hdf4\hdf-eos\gdapi.c(7968) : warning C4701: potentially uninitialized local variable 'maxCol' used c:\programs\gdaltrunk\gdal\frmts\hdf4\hdf-eos\gdapi.c(7970) : warning C4701: potentially uninitialized local variable 'maxRow' used c:\programs\gdaltrunk\gdal\frmts\hdf4\hdf-eos\gdapi.c(8614) : warning C4701: potentially uninitialized local variable 'rank' used
Re: [gdal-dev] editing NetCDF size using GDAL in C++ code
What do you mean by subsets? Do you mean datasets (known as variables in netcdf terminology)? If so, there is no way the netcdf driver can create a netcdf file with subdatasets, sorry. A workaround is to use gdal to create a file for each subdataset and then merge them with the nco commandline tools (e.g. ncks -A file1.nc file2.nc). cheers, Etienne On Thu, Feb 6, 2014 at 10:22 AM, Benedetta Onori benedetta.on...@gmail.comwrote: Hi all, I have a question about NetCDF writing using GDAL. The netcdf driver doesn't support the option GA_Update used to edit dataset, but only GA_ReadOnly. My question is: during the creation of a new netCDF dataset, how can I differentiate the sizes of different subsets? (For example the subset 1 must be 100x100, subset 2 100x1, ...etc...) Thank you for the attention Regards b.o. ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] editing NetCDF size using GDAL in C++ code
For a given dataset, all bands have to be of same dimensions in order to be treated as a single dataset, and not a number of different datasets. This is documented in the GDAL data model page, right on top [1]. A dataset (represented by the GDALDatasethttp://www.gdal.org/classGDALDataset.html class) is an assembly of related raster bands and some information common to them all. In particular the dataset has a concept of the raster size (in pixels and lines) that applies to all the bands. The dataset is also responsible for the georeferencing transform and coordinate system definition of all bands. [1] http://www.gdal.org/gdal_datamodel.html I'm sorry but I don't understand what you mean by In the creation of a NetCDF I treat subsets as bands.. Is this file myfile.nc created by gdal? If the above explanation is not satisfactory, please provide the exact steps you are doing. cheers, Etienne On Thu, Feb 6, 2014 at 1:05 PM, Benedetta Onori benedetta.on...@gmail.comwrote: In the creation of a NetCDF I treat subsets as bands. with gdalinfo I get this: Subdatasets: SUBDATASET_1_NAME=NETCDF:myfile.nc:Band1 SUBDATASET_1_DESC=[1x1] Band1 (64-bit floating-point) SUBDATASET_2_NAME=NETCDF:myfile.nc:Band2 SUBDATASET_2_DESC=[1x1] Band2 (64-bit floating-point) SUBDATASET_3_NAME=NETCDF:myfile.nc:Band3 SUBDATASET_3_DESC=[1x1] Band3 (64-bit floating-point) SUBDATASET_4_NAME=NETCDF:myfile.nc:Band4 SUBDATASET_4_DESC=[1x1] Band4 (64-bit floating-point) gdal-dev@lists.osgeo.org So it recognizes bands as netcdf subsets, but with Create( ) only one size for all bands is possible and not different sizes for different bands-subsets. 2014-02-06 Etienne Tourigny etourigny@gmail.com: What do you mean by subsets? Do you mean datasets (known as variables in netcdf terminology)? If so, there is no way the netcdf driver can create a netcdf file with subdatasets, sorry. A workaround is to use gdal to create a file for each subdataset and then merge them with the nco commandline tools (e.g. ncks -A file1.nc file2.nc ). cheers, Etienne On Thu, Feb 6, 2014 at 10:22 AM, Benedetta Onori benedetta.on...@gmail.com wrote: Hi all, I have a question about NetCDF writing using GDAL. The netcdf driver doesn't support the option GA_Update used to edit dataset, but only GA_ReadOnly. My question is: during the creation of a new netCDF dataset, how can I differentiate the sizes of different subsets? (For example the subset 1 must be 100x100, subset 2 100x1, ...etc...) Thank you for the attention Regards b.o. ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] GDAL and UNITY3D
do you mean the interface that is default in Ubuntu? there should be no problem with that. After all, gdal is a library and a set of command line tools. On Tue, Feb 4, 2014 at 1:36 PM, xavier lhomme lhomme.xav...@gmail.comwrote: Has anyone tried and succeeded to use GDAL with Unity ? Xav ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] Question about gdaladdo and raster mosaicking
As was suggested, you could try using a mask instead. On Thu, Jan 23, 2014 at 7:52 AM, Ammar ammar8...@yahoo.com wrote: Chaitanya, Thank you very much for your reply and suggestion. I have used LZW with predictor 2 compression and added external overviews compressed in LZW too and the image came perfect. The only downside is the size of each of the images has expanded from approx. 2 GB to 18-20 GB. So now I have to choose between quality and size! Attached are two screen shots showing the difference. Thanks again! Regards, Ammar -- *From:* Chaitanya kumar CH chaitanya...@gmail.com *To:* ammar8...@yahoo.com *Cc:* gdal dev gdal-dev@lists.osgeo.org *Sent:* Tuesday, January 21, 2014 6:27 PM *Subject:* Re: Question about gdaladdo and raster mosaicking Ammar, You are using lossy JPEG compression. I'm guessing that the white lines are are actually pixels with values near but not equal to 255. So, they are not treated as nodata pixels. My suggestion is to use another format with lossless compression for the 21 images or use a mask band if you prefer. Let me know how it goes. -- Best regards, Chaitanya Kumar CH On 21-Jan-2014 5:31 pm, ammar8...@yahoo.com wrote: Hello Chaitanya, I was looking online for solutions to my problem and I cam across some of your posts so I thought you might help me with my problem. I have 10,000+ TIFF files that I want to merge into one big Image and serve using GeoServer. I merged every 500 images at a time creating using gdalbuildvrt and gdal_translate along with JPEG compression creating 21 TIFF files. I added overviews then to each of the new 21 files. When adding the images together, once directly as files using QGIS and another time as WMSs using GeoServer, I got white lines/gaps precisely at the border of ever 2 images. I thought the problem is with no data values but I decided to take another approach and merged all the 10,000 files into one huge image and this time I got one perfect image with no lines or gaps. I failed to create overviews for the big image because the process took days or gdaladdo running with no progress. I tested then on smaller images with and without overviews and the ones with no overview came perfect while the one with gdaladdo overviews came with the same problem of lines and gaps. The commands I m using are the following: gdalbuildvrt -srcnodata 255 -vrtnodata 255 -a_srs EPSG:27700 -input_file_list tiff_list.txt mosaic.vrt gdal_translate -of GTiff -co TILED=YES -co BIGTIFF=YES -co COMPRESS=JPEG -co JPEG_QUALITY=80 -co BLOCKXSIZE=512 -co BLOCKYSIZE=512 -co PHOTOMETRIC=YCBCR mosaic.vrt mosaic.tif gdaladdo mosaic.tif -r average --config COMPRESS_OVERVIEW JPEG --config JPEG_QUALITY_OVERVIEW 60 --config INTERLEAVE_OVERVIEW PIXEL --config PHOTOMETRIC_OVERVIEW YCBCR 2 4 8 16 32 64 128 256 512 Any ideas, tips or recommendations would be appreciated! Best regards, Ammar _ Sent from http://osgeo-org.1560.x6.nabble.com ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] gdalinfo coordinates problem?
I noticed that qgis-2.0 also has a problem with this file, but places it around 54 degrees west longitude (if using on-the-fly reprojection to WGS84). The CRS is different from that identified by gdal (the lon_0, units and datum are different) qgis: +proj=lcc +lat_1=40.97 +lat_2=39.93 +lat_0=39.34 +lon_0=-77.75 +x_0=60 +y_0=0 +ellps=GRS80 +towgs84=0,0,0,0,0,0,0 +units=m +no_defs gdal: PROJ.4 : '+proj=lcc +lat_1=40.97 +lat_2=39.93 +lat_0=39.34 +lon_0=-77.75 +x_0=1968500 +y_0=0 +datum=NAD83 +units=us-ft +no_defs ' OGC WKT : PROJCS[NAD_1983_StatePlane_Pennsylvania_South_FIPS_3702_Feet, GEOGCS[NAD83, DATUM[North_American_Datum_1983, SPHEROID[GRS 1980,6378137,298.2572221010002, AUTHORITY[EPSG,7019]], AUTHORITY[EPSG,6269]], PRIMEM[Greenwich,0], UNIT[degree,0.0174532925199433], AUTHORITY[EPSG,4269]], PROJECTION[Lambert_Conformal_Conic_2SP], PARAMETER[standard_parallel_1,40.97], PARAMETER[standard_parallel_2,39.93], PARAMETER[latitude_of_origin,39.34], PARAMETER[central_meridian,-77.75], PARAMETER[false_easting,6458320.41666], PARAMETER[false_northing,0], UNIT[us_survey_feet,0.3048006096012192], AUTHORITY[EPSG,32129]] On Fri, Jan 17, 2014 at 8:37 PM, Even Rouault even.roua...@mines-paris.orgwrote: Le vendredi 17 janvier 2014 21:28:35, Reynolds, Scott a écrit : Hi, I've downloaded several DEMs (e.g 27002570PAS_dem.zip) from ftp://pamap.pasda.psu.edu/pamap_lidar/cycle1/DEM/South/2008/2000/. Gdalinfo is reporting the corner coordinates to be in the vicinity of 91 degrees 31 minutes west longitude which should be more like 75 degrees 36 minutes west longitude. Can someone explain what is happening here? GeoTIFF support for non-meter linear units has been a major source of headaches in GDAL / libgeotiff. People will appreciate looking at frmts/gtiff/gt_wkt_srs.cpp I'm not sure if that GeoTIFF file is correct or not regarding the values of the false easting/northing with respect to the GeoTIFF spec (many in the wild are broken), but when looking at what happens, I noticed that a transformation from meters to feet happened twice. The attached patch fixes that (and the corner coordinates are around 75°), but I'm not sure if it is completely correct in all cases. You should likely create a GDAL Trac ticket with your report and the patch, but I'm not sure who will dare hurting his head against this wall... Even Thanks, Scott -- Geospatial professional services http://even.rouault.free.fr/services.html ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] How to open Erdas Imagine .ige (Large Raster Spill File) Format with gdal
you have not initialized you variable pszFilename, try replacing it with the actual file name of the file, or setting the contents of the variable. On Fri, Dec 13, 2013 at 12:06 AM, Jun Xiong jun.xiong1...@gmail.com wrote: Hi All: I have a Erdas .ige file (=100GB) which was generated by gdal utility gdal_merge.py, then I want to use gdal API to open it, like following: GDALDatasetH hDataset; char *pszFilename; GDALAllRegister(); hDataset = GDALOpen(pszFilename, GA_ReadOnly); The program was compiled successfully, but when I tried to run it, the system prompt Segmentation fault (core dumped) And nothing else, I googled such information and then found very hard to locate the bugs. And ideas? Very appreciated. Jun ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] how to aggregate an image to a lower resolution using gdal (gdalwarp?)
Jan, can you please add your sample dataset and comments in that ticket? Etienne On Tue, Dec 3, 2013 at 6:37 PM, Etienne B. Racine etienn...@gmail.comwrote: I added a ticket http://trac.osgeo.org/gdal/ticket/5311 Etienne 2013/12/2 Etienne Tourigny etourigny@gmail.com The average resampling mode of gdalwarp does average resampling, computes the average of all non-NODATA contributing pixels. It was meant to compute the average of all the pixels in the aggregation window. However, it may have issues in the corners. I am the author of the average and mode algorithms, and I basically copied the aggregation logic from the other algorithms (i.e. which pixels are selected for the aggregation), so there may be something wrong in the logic. Certainly, looking at this simple example shows that something is wrong. I tested average and mode resampling with a fairly large dataset and did not find these problems. Can you create a new bug in the tracker and upload the scripts and data used? I don't have much (any) time to work on this but would be happy to apply a working patch. Cheers, Etienne On Mon, Dec 2, 2013 at 12:22 PM, Etienne B. Racine etienn...@gmail.comwrote: I've tried to dig this a bit. I couldn't understand the logic behind gdal aggregation (or downsampling). I've simplified your example using a smaller raster and deterministic values. Maybe someone could enlighten us by looking at the aggregation values ? Note that the lower right cells values were identical in all dimensions I've tried (about 4 - 10). The example is run on GDAL 1.11dev, released 2013/04/13 Thanks, Etienne # in R: require(raster) filei - '10by10.tif' fileo - '5by5.tif' dm = 4 r - raster(matrix(1:(dm^2), dm, dm)) writeRaster(r, filename=filei, overwrite = TRUE) ## aggregate using R aggregate function using the mean r1 - aggregate(r, fact=2, fun = mean, na.rm = TRUE) file.remove(fileo) cmd - paste(gdalwarp -r average -tr, paste(res(r1), collapse = ), filei, fileo) system(cmd) r2 - raster(fileo) as.matrix(r) [,1] [,2] [,3] [,4] [1,]159 13 [2,]26 10 14 [3,]37 11 15 [4,]48 12 16 lapply(list(r1, r2, r2-r1), as.matrix) [[1]] [,1] [,2] [1,] 3.5 11.5 [2,] 5.5 13.5 [[2]] [,1] [,2] [1,] 6.0 12.0 [2,] 7.5 13.5 [[3]] [,1] [,2] [1,] 2.5 0.5 [2,] 2.0 0.0 2013/8/27 Verbesselt, Jan jan.verbess...@wur.nl Hi all, I have been testing gdalwarp to aggregate (using -r average) an image. In order to better understand what is happening I have created a reproducible example within an R environment and compared it with the aggregate function of the R raster package (see below). There are some differences between the gdalwarp raster (r2) and the aggregated raster (r1). How is the gdalwarp -r average working? Which pixels are selected for averaging (e.g.the corner pixels, center pixels, or all within the aggregation window)? If there is a gdal aggregate option to average pixels comparable to the aggregate function in the R raster package, it would be great as that would potentially faster, and you could also reproject and aggregate at once. Thanks! Jan http://bfast.r-forge.r-project.org http://goo.gl/1mBC5F ## R script require(raster) filei - '10by10.tif' fileo - '5by5.tif' set.seed(123) r - raster(ncols=36, nrows=18) r - setValues(r, round(runif(ncell(r))*10)) r plot(r) writeRaster(r, filename=filei, overwrite = TRUE) ## aggregate using R aggregate function using the mean r1 - aggregate(r, fact=2, fun = mean, na.rm = TRUE) cmd - paste(gdalwarp -r average -tr 20 20 -te -180 -90 180 90 , filei, , fileo, sep = ) system(cmd) r2 - raster(fileo) ## compare plot(r1) plot(r2) r1 r2 getValues(r1) getValues(r2) ## plot(r1-r2) sessionInfo() R sessionInfo() R version 3.0.1 (2013-05-16) Platform: x86_64-pc-linux-gnu (64-bit) locale: [1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C LC_TIME=C LC_COLLATE=C LC_MONETARY=C [6] LC_MESSAGES=CLC_PAPER=C LC_NAME=C LC_ADDRESS=C LC_TELEPHONE=C [11] LC_MEASUREMENT=C LC_IDENTIFICATION=C attached base packages: [1] stats graphics grDevices utils datasets methods base other attached packages: [1] rgdal_0.8-10 raster_2.1-49 sp_1.0-11 loaded via a namespace (and not attached): [1] grid_3.0.1 lattice_0.20-23 tools_3.0.1 rgdal: version: 0.8-10, (SVN revision 478) Geospatial Data Abstraction Library extensions to R successfully loaded Loaded GDAL runtime: GDAL 1.10.0, released 2013/04/24 Path to GDAL shared files: /usr/share/gdal/1.10 Loaded PROJ.4 runtime: Rel. 4.8.0, 6 March 2012, [PJ_VERSION: 480] Path to PROJ.4 shared files: (autodetected) ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] how to aggregate an image to a lower resolution using gdal (gdalwarp?)
I have attached a patch to the ticket that fixes the problem. Anyone interested is welcome to test it. regards, Etienne On Fri, Dec 6, 2013 at 1:58 PM, Etienne B. Racine etienn...@gmail.comwrote: Etienne, I've added the relevant files. I think it's sufficient to debug the function, but if you think it might be helpful, I could add a larger raster and the results of the aggregation. Etienne 2013/12/6 Etienne Tourigny etourigny@gmail.com Jan, can you please add your sample dataset and comments in that ticket? Etienne On Tue, Dec 3, 2013 at 6:37 PM, Etienne B. Racine etienn...@gmail.comwrote: I added a ticket http://trac.osgeo.org/gdal/ticket/5311 Etienne 2013/12/2 Etienne Tourigny etourigny@gmail.com The average resampling mode of gdalwarp does average resampling, computes the average of all non-NODATA contributing pixels. It was meant to compute the average of all the pixels in the aggregation window. However, it may have issues in the corners. I am the author of the average and mode algorithms, and I basically copied the aggregation logic from the other algorithms (i.e. which pixels are selected for the aggregation), so there may be something wrong in the logic. Certainly, looking at this simple example shows that something is wrong. I tested average and mode resampling with a fairly large dataset and did not find these problems. Can you create a new bug in the tracker and upload the scripts and data used? I don't have much (any) time to work on this but would be happy to apply a working patch. Cheers, Etienne On Mon, Dec 2, 2013 at 12:22 PM, Etienne B. Racine etienn...@gmail.com wrote: I've tried to dig this a bit. I couldn't understand the logic behind gdal aggregation (or downsampling). I've simplified your example using a smaller raster and deterministic values. Maybe someone could enlighten us by looking at the aggregation values ? Note that the lower right cells values were identical in all dimensions I've tried (about 4 - 10). The example is run on GDAL 1.11dev, released 2013/04/13 Thanks, Etienne # in R: require(raster) filei - '10by10.tif' fileo - '5by5.tif' dm = 4 r - raster(matrix(1:(dm^2), dm, dm)) writeRaster(r, filename=filei, overwrite = TRUE) ## aggregate using R aggregate function using the mean r1 - aggregate(r, fact=2, fun = mean, na.rm = TRUE) file.remove(fileo) cmd - paste(gdalwarp -r average -tr, paste(res(r1), collapse = ), filei, fileo) system(cmd) r2 - raster(fileo) as.matrix(r) [,1] [,2] [,3] [,4] [1,]159 13 [2,]26 10 14 [3,]37 11 15 [4,]48 12 16 lapply(list(r1, r2, r2-r1), as.matrix) [[1]] [,1] [,2] [1,] 3.5 11.5 [2,] 5.5 13.5 [[2]] [,1] [,2] [1,] 6.0 12.0 [2,] 7.5 13.5 [[3]] [,1] [,2] [1,] 2.5 0.5 [2,] 2.0 0.0 2013/8/27 Verbesselt, Jan jan.verbess...@wur.nl Hi all, I have been testing gdalwarp to aggregate (using -r average) an image. In order to better understand what is happening I have created a reproducible example within an R environment and compared it with the aggregate function of the R raster package (see below). There are some differences between the gdalwarp raster (r2) and the aggregated raster (r1). How is the gdalwarp -r average working? Which pixels are selected for averaging (e.g.the corner pixels, center pixels, or all within the aggregation window)? If there is a gdal aggregate option to average pixels comparable to the aggregate function in the R raster package, it would be great as that would potentially faster, and you could also reproject and aggregate at once. Thanks! Jan http://bfast.r-forge.r-project.org http://goo.gl/1mBC5F ## R script require(raster) filei - '10by10.tif' fileo - '5by5.tif' set.seed(123) r - raster(ncols=36, nrows=18) r - setValues(r, round(runif(ncell(r))*10)) r plot(r) writeRaster(r, filename=filei, overwrite = TRUE) ## aggregate using R aggregate function using the mean r1 - aggregate(r, fact=2, fun = mean, na.rm = TRUE) cmd - paste(gdalwarp -r average -tr 20 20 -te -180 -90 180 90 , filei, , fileo, sep = ) system(cmd) r2 - raster(fileo) ## compare plot(r1) plot(r2) r1 r2 getValues(r1) getValues(r2) ## plot(r1-r2) sessionInfo() R sessionInfo() R version 3.0.1 (2013-05-16) Platform: x86_64-pc-linux-gnu (64-bit) locale: [1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C LC_TIME=C LC_COLLATE=C LC_MONETARY=C [6] LC_MESSAGES=CLC_PAPER=C LC_NAME=C LC_ADDRESS=C LC_TELEPHONE=C [11] LC_MEASUREMENT=C LC_IDENTIFICATION=C attached base packages: [1] stats graphics grDevices utils datasets methods base other attached packages: [1] rgdal_0.8-10 raster_2.1-49 sp_1.0-11 loaded via a namespace (and not attached): [1] grid_3.0.1 lattice_0.20-23 tools_3.0.1 rgdal: version: 0.8-10, (SVN revision 478
Re: [gdal-dev] how to aggregate an image to a lower resolution using gdal (gdalwarp?)
The average resampling mode of gdalwarp does average resampling, computes the average of all non-NODATA contributing pixels. It was meant to compute the average of all the pixels in the aggregation window. However, it may have issues in the corners. I am the author of the average and mode algorithms, and I basically copied the aggregation logic from the other algorithms (i.e. which pixels are selected for the aggregation), so there may be something wrong in the logic. Certainly, looking at this simple example shows that something is wrong. I tested average and mode resampling with a fairly large dataset and did not find these problems. Can you create a new bug in the tracker and upload the scripts and data used? I don't have much (any) time to work on this but would be happy to apply a working patch. Cheers, Etienne On Mon, Dec 2, 2013 at 12:22 PM, Etienne B. Racine etienn...@gmail.comwrote: I've tried to dig this a bit. I couldn't understand the logic behind gdal aggregation (or downsampling). I've simplified your example using a smaller raster and deterministic values. Maybe someone could enlighten us by looking at the aggregation values ? Note that the lower right cells values were identical in all dimensions I've tried (about 4 - 10). The example is run on GDAL 1.11dev, released 2013/04/13 Thanks, Etienne # in R: require(raster) filei - '10by10.tif' fileo - '5by5.tif' dm = 4 r - raster(matrix(1:(dm^2), dm, dm)) writeRaster(r, filename=filei, overwrite = TRUE) ## aggregate using R aggregate function using the mean r1 - aggregate(r, fact=2, fun = mean, na.rm = TRUE) file.remove(fileo) cmd - paste(gdalwarp -r average -tr, paste(res(r1), collapse = ), filei, fileo) system(cmd) r2 - raster(fileo) as.matrix(r) [,1] [,2] [,3] [,4] [1,]159 13 [2,]26 10 14 [3,]37 11 15 [4,]48 12 16 lapply(list(r1, r2, r2-r1), as.matrix) [[1]] [,1] [,2] [1,] 3.5 11.5 [2,] 5.5 13.5 [[2]] [,1] [,2] [1,] 6.0 12.0 [2,] 7.5 13.5 [[3]] [,1] [,2] [1,] 2.5 0.5 [2,] 2.0 0.0 2013/8/27 Verbesselt, Jan jan.verbess...@wur.nl Hi all, I have been testing gdalwarp to aggregate (using -r average) an image. In order to better understand what is happening I have created a reproducible example within an R environment and compared it with the aggregate function of the R raster package (see below). There are some differences between the gdalwarp raster (r2) and the aggregated raster (r1). How is the gdalwarp -r average working? Which pixels are selected for averaging (e.g.the corner pixels, center pixels, or all within the aggregation window)? If there is a gdal aggregate option to average pixels comparable to the aggregate function in the R raster package, it would be great as that would potentially faster, and you could also reproject and aggregate at once. Thanks! Jan http://bfast.r-forge.r-project.org http://goo.gl/1mBC5F ## R script require(raster) filei - '10by10.tif' fileo - '5by5.tif' set.seed(123) r - raster(ncols=36, nrows=18) r - setValues(r, round(runif(ncell(r))*10)) r plot(r) writeRaster(r, filename=filei, overwrite = TRUE) ## aggregate using R aggregate function using the mean r1 - aggregate(r, fact=2, fun = mean, na.rm = TRUE) cmd - paste(gdalwarp -r average -tr 20 20 -te -180 -90 180 90 , filei, , fileo, sep = ) system(cmd) r2 - raster(fileo) ## compare plot(r1) plot(r2) r1 r2 getValues(r1) getValues(r2) ## plot(r1-r2) sessionInfo() R sessionInfo() R version 3.0.1 (2013-05-16) Platform: x86_64-pc-linux-gnu (64-bit) locale: [1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C LC_TIME=C LC_COLLATE=C LC_MONETARY=C [6] LC_MESSAGES=CLC_PAPER=C LC_NAME=C LC_ADDRESS=C LC_TELEPHONE=C [11] LC_MEASUREMENT=C LC_IDENTIFICATION=C attached base packages: [1] stats graphics grDevices utils datasets methods base other attached packages: [1] rgdal_0.8-10 raster_2.1-49 sp_1.0-11 loaded via a namespace (and not attached): [1] grid_3.0.1 lattice_0.20-23 tools_3.0.1 rgdal: version: 0.8-10, (SVN revision 478) Geospatial Data Abstraction Library extensions to R successfully loaded Loaded GDAL runtime: GDAL 1.10.0, released 2013/04/24 Path to GDAL shared files: /usr/share/gdal/1.10 Loaded PROJ.4 runtime: Rel. 4.8.0, 6 March 2012, [PJ_VERSION: 480] Path to PROJ.4 shared files: (autodetected) ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] Using GDALWARP to crop to extremeties of source data
You can use gdal_translte with -projwin or -srcwin parameter On Tue, Nov 26, 2013 at 1:50 PM, JSz ja...@agritech.co.uk wrote: I am using GDAL_grid to create a raster image using the command : *gdal_grid -a invdist:power=2:smoothing=0.001:radius1=0.0005:radius2=0.0005 -of GTiff -zField a_field -ot Float64 -l data data_source.vrt results.tiff* which obviously extrapolates the data beyond the extremities of my data, I can cut the required data with a shape file by using : *gdalwarp -cutline src_shape -crop_to_cutline -srcnodata 0 source.tiff output.tiff* However is there a way to cut to the extremities of my original data without the need for the shape file? Many thanks JSz -- View this message in context: http://osgeo-org.1560.x6.nabble.com/Using-GDALWARP-to-crop-to-extremeties-of-source-data-tp5091315.html Sent from the GDAL - Dev mailing list archive at Nabble.com. ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] WRF NetCDF import
On Tue, Nov 26, 2013 at 8:32 AM, Lee Eddington lee.w.edding...@gmail.comwrote: Roger, Attached is the script I'm using. Here are the results from gdalinfo on the output file: lees-mbp:full Lee$ gdalinfo u10_2013112018.tif Driver: GTiff/GeoTIFF Files: u10_2013112018.tif Size is 198, 153 Coordinate System is: PROJCS[unnamed, GEOGCS[unknown, DATUM[unknown, SPHEROID[unretrievable - using WGS84,6378137,298.257223563]], PRIMEM[Greenwich,0], UNIT[degree,0.0174532925199433]], PROJECTION[Mercator_1SP], PARAMETER[central_meridian,115.02], PARAMETER[scale_factor,0.9893189261265199], PARAMETER[false_easting,0], PARAMETER[false_northing,0], UNIT[metre,1, AUTHORITY[EPSG,9001]]] Origin = (-295898.731194700172637,-696504.655801336281002) Pixel Size = (2988.872338323183612,-2964.865850256682734) Metadata: AREA_OR_POINT=Area Image Structure Metadata: INTERLEAVE=BAND Corner Coordinates: Upper Left ( -295898.731, -696504.656) (112d19'59.51E, 6d21'13.48S) Lower Left ( -295898.731,-1150129.131) (112d19'59.51E, 10d27'15.97S) Upper Right ( 295897.992, -696504.656) (117d42'24.46E, 6d21'13.48S) Lower Right ( 295897.992,-1150129.131) (117d42'24.46E, 10d27'15.97S) Center ( -0.370, -923316.893) (115d 1'11.99E, 8d24'34.51S) Band 1 Block=198x10 Type=Float32, ColorInterp=Gray When I import this into GRASS I get coordinates of meters instead of lat/lon degrees. I want to keep the mercator projection of the model and not convert to a lat/lon projection. Mercator projection is by definition in linear units (meters, km, whatever), not angular units (degrees). Lee On Tue, Nov 26, 2013 at 1:45 AM, Roger Veciana i Rovira rveci...@gmail.com wrote: Lee, If I understand what are you saiying, you have to define a proper projection first: proj_out = osr.SpatialReference() proj_out.ImportFromEPSG(4326) Which will create the lat lon projection you need (using WGS84 datum, maybe this won't be exact in your model). Then, you have to set the geotransform directly in degrees as you want. Roger 2013/11/26 Lee Eddington lee.w.edding...@gmail.com I think I might be getting close. I found the following which I'm hoping will create the GeoTiff with angular units of degrees instead of linear units of meters: OGRErr OGRSpatialReference::SetAngularUnits(const char * pszUnitsName, double dfInRadians ) and tried to use it, but don't know the proper pszUnitsName. I've tried degree, degrees, DEGREE, DEGREES, DEG, SRS_UA_DEGREE, but always get a return code of 6 instead of 0. I can't find a list of names or any examples of how to use this properly. On Mon, Nov 25, 2013 at 2:00 PM, Lee Eddington lee.w.edding...@gmail.com wrote: Roger, Thank you for the info and link. Using a modified version of your code I was able to create and import into GRASS a GeoTIFF with the proper georeferencing. The coordinate system units are in meters though, and I'd prefer to have them in degrees lat/lon. Do you know if there's a way to specify that using the oar or gdal routines in your code? Thanks, Lee On Mon, Nov 25, 2013 at 10:23 AM, Etienne Tourigny etourigny@gmail.com wrote: Forwarding to the list and others... -- Forwarded message -- From: Roger Veciana i Rovira rveci...@gmail.com Date: Mon, Nov 25, 2013 at 9:35 AM Subject: Re: [gdal-dev] WRF NetCDF import To: Etienne Tourigny etourigny@gmail.com As Etienne told you, the geotransform and projection from your files is not in the format that GDAL can understand. I wrote a script that reads the projection and calculates the geotransform to save it in a GeoTIFF file: http://geoexamples.blogspot.com.es/2013/09/reading-wrf-netcdf-files-with-gdal.html The script also interpolates the WRF sigma levels to the desired pressure level, so you will have to look for the functions that you need. Anyway, the solution is quite complicated, and using cdo, nco or other tools (I use ARWPost) may be easier. Roger 2013/11/24 Etienne Tourigny etourigny@gmail.com The georeferencing info you refer to is non-standard, and GDAL cannot use that. It (and presumably the grass provider, although I don't know) use the CF standard for encoding lat/lon information. what is the output of gdalinfo NETCDF:test_full.nc:HGT I don't know how the grass provider deals with subdatasets, but if you open the file in qgis (using the gdal provider, it will sllow you to select the subdataset you want). You could also extract the variable with another tool like cdo or nco. Etienne On Sat, Nov 23, 2013 at 7:47 PM, Lee Eddington lee.w.edding...@gmail.com wrote: I'm trying to use r.in.gdal in GRASS GIS to import Weather Research Forecasting (WRF) model NetCDF output. There have been some tips provided by the GRASS users list, but none have worked for me. I can import the data as a simple x,y array
[gdal-dev] Fwd: WRF NetCDF import
Forwarding to the list and others... -- Forwarded message -- From: Roger Veciana i Rovira rveci...@gmail.com Date: Mon, Nov 25, 2013 at 9:35 AM Subject: Re: [gdal-dev] WRF NetCDF import To: Etienne Tourigny etourigny@gmail.com As Etienne told you, the geotransform and projection from your files is not in the format that GDAL can understand. I wrote a script that reads the projection and calculates the geotransform to save it in a GeoTIFF file: http://geoexamples.blogspot.com.es/2013/09/reading-wrf-netcdf-files-with-gdal.html The script also interpolates the WRF sigma levels to the desired pressure level, so you will have to look for the functions that you need. Anyway, the solution is quite complicated, and using cdo, nco or other tools (I use ARWPost) may be easier. Roger 2013/11/24 Etienne Tourigny etourigny@gmail.com The georeferencing info you refer to is non-standard, and GDAL cannot use that. It (and presumably the grass provider, although I don't know) use the CF standard for encoding lat/lon information. what is the output of gdalinfo NETCDF:test_full.nc:HGT I don't know how the grass provider deals with subdatasets, but if you open the file in qgis (using the gdal provider, it will sllow you to select the subdataset you want). You could also extract the variable with another tool like cdo or nco. Etienne On Sat, Nov 23, 2013 at 7:47 PM, Lee Eddington lee.w.edding...@gmail.comwrote: I'm trying to use r.in.gdal in GRASS GIS to import Weather Research Forecasting (WRF) model NetCDF output. There have been some tips provided by the GRASS users list, but none have worked for me. I can import the data as a simple x,y array with no georeferencing, but information about georeferencing is in the file as other programs read, georeference and display the file correctly. gdalinfo produces the following: lees-mbp:full Lee$ gdalinfo test_full.nc Warning 1: No UNIDATA NC_GLOBAL:Conventions attribute Driver: netCDF/Network Common Data Format Files: test_full.nc Size is 512, 512 Coordinate System is `' Metadata: NC_GLOBAL#BL_PBL_PHYSICS=1 NC_GLOBAL#BOTTOM-TOP_GRID_DIMENSION=28 NC_GLOBAL#BOTTOM-TOP_PATCH_END_STAG=28 NC_GLOBAL#BOTTOM-TOP_PATCH_END_UNSTAG=27 NC_GLOBAL#BOTTOM-TOP_PATCH_START_STAG=1 NC_GLOBAL#BOTTOM-TOP_PATCH_START_UNSTAG=1 NC_GLOBAL#BUCKET_J=-1 NC_GLOBAL#BUCKET_MM=-1 NC_GLOBAL#CEN_LAT=-8.4095154 NC_GLOBAL#CEN_LON=115.02 NC_GLOBAL#CU_PHYSICS=0 NC_GLOBAL#DAMP_OPT=0 NC_GLOBAL#DAMPCOEF=0.2 NC_GLOBAL#DFI_OPT=0 NC_GLOBAL#DIFF_6TH_FACTOR=0.12 NC_GLOBAL#DIFF_6TH_OPT=0 NC_GLOBAL#DIFF_OPT=1 NC_GLOBAL#DT=16.66 NC_GLOBAL#DX=3000 NC_GLOBAL#DY=3000 NC_GLOBAL#FEEDBACK=0 NC_GLOBAL#GFDDA_END_H=0 NC_GLOBAL#GFDDA_INTERVAL_M=0 NC_GLOBAL#GMT=12 NC_GLOBAL#GRID_FDDA=0 NC_GLOBAL#GRID_ID=3 NC_GLOBAL#GRID_SFDDA=0 NC_GLOBAL#GRIDTYPE=C NC_GLOBAL#HYPSOMETRIC_OPT=2 NC_GLOBAL#I_PARENT_START=34 NC_GLOBAL#ISFTCFLX=0 NC_GLOBAL#ISHALLOW=0 NC_GLOBAL#ISICE=24 NC_GLOBAL#ISLAKE=-1 NC_GLOBAL#ISOILWATER=14 NC_GLOBAL#ISURBAN=1 NC_GLOBAL#ISWATER=16 NC_GLOBAL#J_PARENT_START=34 NC_GLOBAL#JULDAY=324 NC_GLOBAL#JULYR=2013 NC_GLOBAL#KHDIF=0 NC_GLOBAL#KM_OPT=4 NC_GLOBAL#KVDIF=0 NC_GLOBAL#MAP_PROJ=3 NC_GLOBAL#MFSHCONV=0 NC_GLOBAL#MMINLU=USGS NC_GLOBAL#MOAD_CEN_LAT=-8.4095078 NC_GLOBAL#MOIST_ADV_OPT=1 NC_GLOBAL#MP_PHYSICS=3 NC_GLOBAL#NUM_LAND_CAT=24 NC_GLOBAL#OBS_NUDGE_OPT=0 NC_GLOBAL#OMLCALL=0 NC_GLOBAL#PARENT_GRID_RATIO=3 NC_GLOBAL#PARENT_ID=2 NC_GLOBAL#POLE_LAT=90 NC_GLOBAL#POLE_LON=0 NC_GLOBAL#PREC_ACC_DT=0 NC_GLOBAL#RA_LW_PHYSICS=1 NC_GLOBAL#RA_SW_PHYSICS=1 NC_GLOBAL#SCALAR_ADV_OPT=1 NC_GLOBAL#SF_SFCLAY_PHYSICS=1 NC_GLOBAL#SF_SURFACE_PHYSICS=2 NC_GLOBAL#SF_URBAN_PHYSICS=0 NC_GLOBAL#SGFDDA_END_H=0 NC_GLOBAL#SGFDDA_INTERVAL_M=0 NC_GLOBAL#SHCU_PHYSICS=0 NC_GLOBAL#SIMULATION_START_DATE=2013-11-20_12:00:00 NC_GLOBAL#SMOOTH_OPTION=0 NC_GLOBAL#SOUTH-NORTH_GRID_DIMENSION=154 NC_GLOBAL#SOUTH-NORTH_PATCH_END_STAG=154 NC_GLOBAL#SOUTH-NORTH_PATCH_END_UNSTAG=153 NC_GLOBAL#SOUTH-NORTH_PATCH_START_STAG=1 NC_GLOBAL#SOUTH-NORTH_PATCH_START_UNSTAG=1 NC_GLOBAL#SST_UPDATE=0 NC_GLOBAL#STAND_LON=115.02 NC_GLOBAL#START_DATE=2013-11-20_12:00:00 NC_GLOBAL#SURFACE_INPUT_SOURCE=1 NC_GLOBAL#SWRAD_SCAT=1 NC_GLOBAL#TITLE= OUTPUT FROM WRF V3.4 MODEL NC_GLOBAL#TKE_ADV_OPT=1 NC_GLOBAL#TRUELAT1=-8.4095001 NC_GLOBAL#TRUELAT2=0 NC_GLOBAL#W_DAMPING=0 NC_GLOBAL#WEST-EAST_GRID_DIMENSION=199 NC_GLOBAL#WEST-EAST_PATCH_END_STAG=199 NC_GLOBAL#WEST-EAST_PATCH_END_UNSTAG=198 NC_GLOBAL#WEST-EAST_PATCH_START_STAG=1 NC_GLOBAL#WEST-EAST_PATCH_START_UNSTAG=1 Subdatasets: SUBDATASET_1_NAME=NETCDF:test_full.nc:Times SUBDATASET_1_DESC=[1x19] Times (8-bit character) SUBDATASET_2_NAME=NETCDF:test_full.nc:LU_INDEX SUBDATASET_2_DESC=[1x153x198] LU_INDEX
Re: [gdal-dev] reading hdf4 files in R
fwtool is really old so you should not use it and uninstall it. You may want to use the R package Rgdal, but I have not used iit under windows. http://cran.r-project.org/web/packages/rgdal/index.html On Mon, Nov 25, 2013 at 6:16 AM, Lucia Rueda Ramirez lucia.ru...@ba.ieo.eswrote: Hi, I am trying to read hdf4 files in R. I have tried to install the gdal package from http://trac.osgeo.org/gdal/wiki/DownloadSource but I get an error message saying that the description file is missing (Error: Does not appear to be an R package (no DESCRIPTION)). I have also tried to load the hdf4 files using the modis library and the code sds - getSds(C:/mypath/T20011612001168.L3m_8D_CHL_chlor_a_4km.hdf) But I get the following errors: 'FWTools/OSGeo4W' installation not found or path not set. If you don't have installed one of them you can get it from ' http://fwtools.maptools.org/' or 'http://trac.osgeo.org/osgeo4w/' (recommanded) Trying to autodetect path to 'FWTools/OSGeo4W' (this may takes some time, you can interupt this process and set it manually, see 'gdalPath' argument in '?MODISoptions': Found 'FWTools' version: '1.7.0b2' to enalbe this run: MODISoptions(gdalPath='c:/Archivos de programa/FWTools2.4.7/bin') Found 'OSgeo4W' version: '1:' in 'c:/OSGeo4W/bin' but without HDF4 support...strange, try to remove and re-install 'OSgeo4W'! Error in getSds(C:/Documents and Settings/lucia.rueda/Mis documentos/MEDITS/tema_clorofila/T20011612001168.L3m_8D_CHL_chlor_a_4km.hdf) : Method GDAL does not work. Is GDAL installed properly on your system? Run: 'checkTools()' to check out which metods should work on your system! I have already installed the FWTools/OSGeo4W from the recommended url above though I got installation errors as well... Anyone with the same problem?? Any help would be apreciated. I have gone through most forums but can't figure out how to solve this. Thanks a lot Lucia ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] WRF NetCDF import
The georeferencing info you refer to is non-standard, and GDAL cannot use that. It (and presumably the grass provider, although I don't know) use the CF standard for encoding lat/lon information. what is the output of gdalinfo NETCDF:test_full.nc:HGT I don't know how the grass provider deals with subdatasets, but if you open the file in qgis (using the gdal provider, it will sllow you to select the subdataset you want). You could also extract the variable with another tool like cdo or nco. Etienne On Sat, Nov 23, 2013 at 7:47 PM, Lee Eddington lee.w.edding...@gmail.comwrote: I'm trying to use r.in.gdal in GRASS GIS to import Weather Research Forecasting (WRF) model NetCDF output. There have been some tips provided by the GRASS users list, but none have worked for me. I can import the data as a simple x,y array with no georeferencing, but information about georeferencing is in the file as other programs read, georeference and display the file correctly. gdalinfo produces the following: lees-mbp:full Lee$ gdalinfo test_full.nc Warning 1: No UNIDATA NC_GLOBAL:Conventions attribute Driver: netCDF/Network Common Data Format Files: test_full.nc Size is 512, 512 Coordinate System is `' Metadata: NC_GLOBAL#BL_PBL_PHYSICS=1 NC_GLOBAL#BOTTOM-TOP_GRID_DIMENSION=28 NC_GLOBAL#BOTTOM-TOP_PATCH_END_STAG=28 NC_GLOBAL#BOTTOM-TOP_PATCH_END_UNSTAG=27 NC_GLOBAL#BOTTOM-TOP_PATCH_START_STAG=1 NC_GLOBAL#BOTTOM-TOP_PATCH_START_UNSTAG=1 NC_GLOBAL#BUCKET_J=-1 NC_GLOBAL#BUCKET_MM=-1 NC_GLOBAL#CEN_LAT=-8.4095154 NC_GLOBAL#CEN_LON=115.02 NC_GLOBAL#CU_PHYSICS=0 NC_GLOBAL#DAMP_OPT=0 NC_GLOBAL#DAMPCOEF=0.2 NC_GLOBAL#DFI_OPT=0 NC_GLOBAL#DIFF_6TH_FACTOR=0.12 NC_GLOBAL#DIFF_6TH_OPT=0 NC_GLOBAL#DIFF_OPT=1 NC_GLOBAL#DT=16.66 NC_GLOBAL#DX=3000 NC_GLOBAL#DY=3000 NC_GLOBAL#FEEDBACK=0 NC_GLOBAL#GFDDA_END_H=0 NC_GLOBAL#GFDDA_INTERVAL_M=0 NC_GLOBAL#GMT=12 NC_GLOBAL#GRID_FDDA=0 NC_GLOBAL#GRID_ID=3 NC_GLOBAL#GRID_SFDDA=0 NC_GLOBAL#GRIDTYPE=C NC_GLOBAL#HYPSOMETRIC_OPT=2 NC_GLOBAL#I_PARENT_START=34 NC_GLOBAL#ISFTCFLX=0 NC_GLOBAL#ISHALLOW=0 NC_GLOBAL#ISICE=24 NC_GLOBAL#ISLAKE=-1 NC_GLOBAL#ISOILWATER=14 NC_GLOBAL#ISURBAN=1 NC_GLOBAL#ISWATER=16 NC_GLOBAL#J_PARENT_START=34 NC_GLOBAL#JULDAY=324 NC_GLOBAL#JULYR=2013 NC_GLOBAL#KHDIF=0 NC_GLOBAL#KM_OPT=4 NC_GLOBAL#KVDIF=0 NC_GLOBAL#MAP_PROJ=3 NC_GLOBAL#MFSHCONV=0 NC_GLOBAL#MMINLU=USGS NC_GLOBAL#MOAD_CEN_LAT=-8.4095078 NC_GLOBAL#MOIST_ADV_OPT=1 NC_GLOBAL#MP_PHYSICS=3 NC_GLOBAL#NUM_LAND_CAT=24 NC_GLOBAL#OBS_NUDGE_OPT=0 NC_GLOBAL#OMLCALL=0 NC_GLOBAL#PARENT_GRID_RATIO=3 NC_GLOBAL#PARENT_ID=2 NC_GLOBAL#POLE_LAT=90 NC_GLOBAL#POLE_LON=0 NC_GLOBAL#PREC_ACC_DT=0 NC_GLOBAL#RA_LW_PHYSICS=1 NC_GLOBAL#RA_SW_PHYSICS=1 NC_GLOBAL#SCALAR_ADV_OPT=1 NC_GLOBAL#SF_SFCLAY_PHYSICS=1 NC_GLOBAL#SF_SURFACE_PHYSICS=2 NC_GLOBAL#SF_URBAN_PHYSICS=0 NC_GLOBAL#SGFDDA_END_H=0 NC_GLOBAL#SGFDDA_INTERVAL_M=0 NC_GLOBAL#SHCU_PHYSICS=0 NC_GLOBAL#SIMULATION_START_DATE=2013-11-20_12:00:00 NC_GLOBAL#SMOOTH_OPTION=0 NC_GLOBAL#SOUTH-NORTH_GRID_DIMENSION=154 NC_GLOBAL#SOUTH-NORTH_PATCH_END_STAG=154 NC_GLOBAL#SOUTH-NORTH_PATCH_END_UNSTAG=153 NC_GLOBAL#SOUTH-NORTH_PATCH_START_STAG=1 NC_GLOBAL#SOUTH-NORTH_PATCH_START_UNSTAG=1 NC_GLOBAL#SST_UPDATE=0 NC_GLOBAL#STAND_LON=115.02 NC_GLOBAL#START_DATE=2013-11-20_12:00:00 NC_GLOBAL#SURFACE_INPUT_SOURCE=1 NC_GLOBAL#SWRAD_SCAT=1 NC_GLOBAL#TITLE= OUTPUT FROM WRF V3.4 MODEL NC_GLOBAL#TKE_ADV_OPT=1 NC_GLOBAL#TRUELAT1=-8.4095001 NC_GLOBAL#TRUELAT2=0 NC_GLOBAL#W_DAMPING=0 NC_GLOBAL#WEST-EAST_GRID_DIMENSION=199 NC_GLOBAL#WEST-EAST_PATCH_END_STAG=199 NC_GLOBAL#WEST-EAST_PATCH_END_UNSTAG=198 NC_GLOBAL#WEST-EAST_PATCH_START_STAG=1 NC_GLOBAL#WEST-EAST_PATCH_START_UNSTAG=1 Subdatasets: SUBDATASET_1_NAME=NETCDF:test_full.nc:Times SUBDATASET_1_DESC=[1x19] Times (8-bit character) SUBDATASET_2_NAME=NETCDF:test_full.nc:LU_INDEX SUBDATASET_2_DESC=[1x153x198] LU_INDEX (32-bit floating-point) (more SUBDATASETs) SUBDATASET_106_NAME=NETCDF:test_full.nc:LANDMASK SUBDATASET_106_DESC=[1x153x198] LANDMASK (32-bit floating-point) SUBDATASET_107_NAME=NETCDF:test_full.nc:SST SUBDATASET_107_DESC=[1x153x198] SST (32-bit floating-point) Corner Coordinates: Upper Left (0.0,0.0) Lower Left (0.0, 512.0) Upper Right ( 512.0,0.0) Lower Right ( 512.0, 512.0) Center ( 256.0, 256.0) My first question is why am I getting the following warning?: Warning 1: No UNIDATA NC_GLOBAL:Conventions attribute Next, I see what appears is georeferencing info in the following lines: NC_GLOBAL#CEN_LAT=-8.4095154 NC_GLOBAL#CEN_LON=115.02 NC_GLOBAL#MAP_PROJ=3 (Mercator) NC_GLOBAL#MOAD_CEN_LAT=-8.4095078 NC_GLOBAL#STAND_LON=115.02 NC_GLOBAL#TRUELAT1=-8.4095001 but obviously this is not being
Re: [gdal-dev] RFC 44: JSON/XML output for ogrinfo/gdalinfo
You might think of adding json/xml output also to gdalsrsinfo, it should not be much work. For example, if you have a srs node output by gdalsrsinfo, you could have the same (and only) node in gdalsrsinfo. Also consider that there are various output formats for srs (mostly in gdalsrsinfo, but also proj4 for gdalinfo). cheers Etienne On Tue, Nov 19, 2013 at 2:42 PM, Howard Butler how...@hobu.co wrote: All, Dan Little and myself would like to put forward an RFC proposing -xml and -json output support for both ogrinfo and gdalinfo. No changes are proposed to the current text format (which would continue to be the default output format), but having JSON and XML available would greatly ease the integration of gdalinfo and ogrinfo into existing processing workflows without requiring that someone jump into scripting land. Though it might have been overkill to have an RFC, I thought it would be useful to have something to collaborate on if others have ideas about these particular features. http://trac.osgeo.org/gdal/wiki/rfc44_gdalinfoxml We look forward to your feedback, Howard ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] gdal get geolocation array in HDF5 dataset
geolocation is stored in the GEOLOCATION metadata domain, so this should probably work: GDALGetMetadata( dataset, GEOLOCATION ) see http://trac.osgeo.org/gdal/wiki/rfc4_geolocate On Wed, Oct 30, 2013 at 6:38 AM, laura0 lauram...@iol.it wrote: Hi all, I would like to know I can I read with gdal the geolocation array stored inside an O3MSAF HDF5 data. With GDALGetMetadata( dataset, ); I get the list of metadata (es. Fill Value, Title, Unit but not the array with the latitude and longitude values), with GDALGetMetadata( dataset, SUBDATASETS ); just the list of SUBDATASETS: all the DATA variable, GEOLOCATION are excluded. Suggestions? Thanks, Laura -- View this message in context: http://osgeo-org.1560.x6.nabble.com/gdal-get-geolocation-array-in-HDF5-dataset-tp5086505.html Sent from the GDAL - Dev mailing list archive at Nabble.com. ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] GDAL 1.10.1 requirements
gdal-1.10.1 makes use of the 'ncconfig' utility which is available since netcdf-4.1.3 (if I remember correctly). In any case, it works with older versions (e.g. 4,1,1, 3.6.x) but cannot detect netcdf-4 support as it falls back to the previous method for netcdf detection. As Even mentioned, building with netcdf-3.6.x still works.In some case, it requires the use of the --with-netcdf-some_dir command switch to configure. Are you sure you have the netcdf-dev package (or equivalent) installed? Etienne On Mon, Oct 14, 2013 at 5:54 AM, laura0 lauram...@iol.it wrote: Hi all, I would like to know which versions of netcdf, hdf4, hdf5 libraries are supported by the new release of GDAL. I tried for instance 4.1.1 netcdf library but, after ./configure, I see netcdf supported NO . So I think that version is too old. With the latest versions (netcdf 4.3.0, hdf5 1.8.11 , hdf4 4.2.9 ) everything looks going fine but processing an hdf5 dataset I got the following error: HDF5-DIAG: Error detected in HDF5 (1.8.11) thread 0: #000: H5A.c line 557 in H5Aopen(): unable to load attribute info from object header for attribute: 'PRODUCT_NAME' major: Attribute I was able to open the same dataset instead with gdal 1.9 and library versions: hdf5 1.8.9 , hdf4 4.2.8 , netcdf 4.2.1.1 Is it a problem of versions or something else? Many thanks, regards Laura -- View this message in context: http://osgeo-org.1560.x6.nabble.com/GDAL-1-10-1-requirements-tp5083369.html Sent from the GDAL - Dev mailing list archive at Nabble.com. ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] GTiff stack to NetCDF
what you want is probably outside the scope of gdal. It would require some clever metadata management so that gdal_translate puts them in a single file... I would advise you convert them all to netcdf using gdal_translate and then use python-netcdf4 (not the one from numpy/scipy) to stack them in the temporal dimension. On Mon, Sep 2, 2013 at 7:09 PM, David Shean dsh...@gmail.com wrote: Hi list, I'm trying to package a timeseries of GTiff rasters with identical projection/extent/resolution as a single NetCDF file for distribution. I've spent the past hour consulting the online doc and playing with gdal_translate, gdalbuildvrt and gdalwarp without any success. Is there an easy way to do this using existing gdal command line utilities? I figured I'd ask before resorting to a custom solution using the NetCDF Python API. Thanks. -David ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] netCDF Data without subdatasets
The file has no subdatasets, so you can use any of the gdal tools
(gdalwarp, gdal_translate) normally. If you want a gtiff file you
would use gdal_translate
gdal_translate -of gtiff in.nc out.tif
On Tue, Aug 6, 2013 at 11:38 AM, Gunnar Oehmichen
oehm8...@uni-landau.de wrote:
Hi,
this is my first time using netcdf data and gdal library on Ubuntu 12.04
I have downloaded a subsetted monthly precipitation dataset from:
http://www.esrl.noaa.gov/psd/data/gridded/data.cmap.html
I want to create timelines of a few gridcells (covering germany)
As I understand, before converting bands to raster-images by gdal_translate
one has to extract information from the subdatasets.
My Problem is:
using
gdalinfo X139.14.20.244.217.3.48.41.nc gdalinfo.txt
No subdatasets appear. I would have expected something like to the follwing
to be included in the output of gdalinfo:
SUBDATASET_1_NAME=NETCDF:...
SUBDATASET_1_DESC=...
SUBDATASET_2_NAME=...
SUBDATASET_2_DESC=...
SUBDATASET_3_NAME=...
SUBDATASET_3_DESC=...
Instead the output is as following:
Driver: netCDF/Network Common Data Format
Files: X139.14.20.244.217.3.47.29.nc
Size is 6, 5
Coordinate System is `'
Origin = (2.500,57.500)
Pixel Size = (2.500,-2.500)
Metadata:
lat#actual_range={ 56.25, 46.25 }
lat#axis=Y
lat#long_name=Latitude
lat#standard_name=latitude
lat#units=degrees_north
lon#actual_range={ 3.75, 16.25 }
lon#axis=X
lon#long_name=Longitude
lon#standard_name=longitude
lon#units=degrees_east
NC_GLOBAL#Conventions=COARDS
NC_GLOBAL#documentation=http://www.esrl.noaa.gov/psd/data/gridded/data.cmap.html
NC_GLOBAL#history=Tue Aug 6 03:47:30 2013: ncks -O -d
lat,46.00,57.00 -d lon,3.00,17.00 -d time,0,2401
/Datasets/cmap/std/precip.pentad.mean.nc
/Public/www/X139.14.20.244.217.3.47.29.nc
Created by CAS 09/2012 using Version 1201
NC_GLOBAL#platform=Analyses
NC_GLOBAL#references=http://www.esrl.noaa.gov/psd/data/gridded/data.cmap.html
NC_GLOBAL#source=ftp ftp.cpc.ncep.noaa.gov precip/cmap/pentad
NC_GLOBAL#title=CPC Merged Analysis of Precipitation Pentad data (excludes
NCEP Reanalysis I)
NC_GLOBAL#Version=V1201
precip#actual_range={ 0, 25.07 }
precip#add_offset=0
precip#dataset=CPC Merged Analysis of Precipitation Standard
precip#least_significant_digit=2
precip#level_desc=Surface
precip#long_name=Average Pentad Rate of Precipitation
precip#missing_value=-9.96921e+36
precip#parent_stat=Mean
precip#precision=2
precip#scale_factor=1
precip#statistic=Pentad Mean
precip#units=mm/day
precip#valid_range={ 0, 250 }
precip#var_desc=Precipitation
time#actual_range={ 1569072, 1857384 }
time#avg_period=-01-00 00:00:00
time#axis=T
time#delta_t=-00-05 00:00:00
time#long_name=Time
time#standard_name=time
time#units=hours since 1800-01-01 00:00:0.0
Corner Coordinates:
Upper Left ( 2.500, 57.500)
Lower Left ( 2.500, 45.000)
Upper Right ( 17.500, 57.500)
Lower Right ( 17.500, 45.000)
Center ( 10.000, 51.250)
Band 1 Block=6x1 Type=Float32, ColorInterp=Undefined
NoData Value=-9.96920996838686905e+36
Metadata:
actual_range={ 0, 25.07 }
add_offset=0
dataset=CPC Merged Analysis of Precipitation Standard
least_significant_digit=2
level_desc=Surface
long_name=Average Pentad Rate of Precipitation
missing_value=-9.96921e+36
NETCDF_DIMENSION_time=1569072
NETCDF_time_units=hours since 1800-01-01 00:00:0.0
NETCDF_VARNAME=precip
parent_stat=Mean
precision=2
scale_factor=1
statistic=Pentad Mean
units=mm/day
valid_range={ 0, 250 }
var_desc=Precipitation
And the next bands following.
---
Here is the metadata:
---
Metadata for /Public/www/X139.14.20.244.217.3.47.29.nc
Dimensions:
time2402
lat 5
lon 6
precip 2402 5 6
Variables:
time
long_name Time
avg_period -01-00 00:00:00
axisT
standard_name time
delta_t -00-05 00:00:00
actual_range1979-01-01 00:00:00 2011-11-22
00:00:00
units hours since 1800-01-01 00:00:0.0
lat
long_name Latitude
axisY
standard_name latitude
actual_range56.25 46.25
units degrees_north
lon
long_name Longitude
axisX
standard_name longitude
Re: [gdal-dev] How do I tell if an OGRSpatialReference has been initialized?
You could try calling exportToWkt() and test if the string is null or empty (although I haven't tested that) - make sure to free the string afterwards. Etienne On Mon, Jul 22, 2013 at 8:44 PM, David Strip g...@stripfamily.net wrote: Given an OGRSpatialReference class object, how do I tell if it's been initialized to anything? (ie,clear() was called or else was constructed with a null string an no further action was taken to set the SRS?) I've looked over the interface and can't spot anything that tells me it's in a clear state, but of course I might have missed it. Thanks David __**_ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/**mailman/listinfo/gdal-devhttp://lists.osgeo.org/mailman/listinfo/gdal-dev ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] GRIB2 to NetCDF - Large areas of data missing
You probably are experiencing a known bug, when the warping operation needs more memory than the warp cache. Try setting adding the following to the gdalwarp command (if it doesn't work try a larger value). --config GDAL_CACHEMAX 1000 -wm 1000 Also please update the ticket with any new information http://trac.osgeo.org/gdal/ticket/4484 cheers Etienne On Wed, Jul 17, 2013 at 5:31 PM, Jason Greenlaw - NOAA Affiliate jason.green...@noaa.gov wrote: I am experiencing strange behavior when reprojecting/converting a GRIB2 file to NetCDF using gdalwarp, where the resulting NetCDF file is missing lots of data (or possibly being set incorrectly to NoData) west of a certain longitude. However, if I instead reproject from GRIB2 to an intermediate format (e.g. GeoTIFF), then convert to NetCDF using gdal_translate, the result is correct. The source projection is the weather service's custom lambert conformal, and the destination is WGS84. I'm using the following command: gdalwarp -wm 256 -of netcdf -srcnodata .0 -dstnodata .0 -t_srs EPSG:4326 conus-wspd-01.bin conus-wspd-01.nc GDAL versions tested were 1.9.2 and 1.10. Screenshot of original GRIB2 file (first GRIB message) as displayed in ArcMap: http://jetstreak.unh.edu/gdal/full.png Screenshot of resulting NetCDF file (variable Band1) as displayed in ncBrowse: http://jetstreak.unh.edu/gdal/cropped.png Original GRIB2 file (6.3mb): http://jetstreak.unh.edu/gdal/conus-wspd-01.bin Resulting NetCDF file (171mb): http://jetstreak.unh.edu/gdal/conus-wspd-01.nc Anyone run into a similar problem or have any ideas for how to resolve this? Thanks, Jason ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] hillshading to vector
On Thu, Jul 4, 2013 at 3:58 AM, Carlo A. Bertelli (Charta s.r.l.) berte...@charta.acme.com wrote: Thanks a lot, Roger, it worked like a charm, just fiddling a little with parameters and than run it in python. It's really magic, I struggled with this question for two days before writing to the list. I think that instead of making another script you could add it as an option to gdal_contour.py and use it alternatively with -l (default) or -p (output to polygons). not so sure about that, because the script requires matplotlib (although the option could be disabled if matplotlib cannot be imported). By the way, another enhancement for gdal_contour could be adding an options to cut the output with a line or polygon layer (say the shoreline or other boundary). c c On Wed, Jul 3, 2013 at 8:04 PM, Roger Veciana i Rovira rveci...@gmail.com wrote: Hi, I was just working on that. I think that you need a script like gdal_contour, but creatig polygons instead of lines. The algorithm used is Marching Squares. You can see the code, without explanation, here: https://github.com/rveciana/geoexamples/tree/master/python/raster_isobands In some weeks I'll add some more info in a README file and transform the file into a working script, not just a function. But you can use it just changing the file name and the intervals. Roger ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] g++ compilation errors when trying to compile GDAL on Fedora 17 64bit
It seems you don't have c++ installed... you can *probably* install it like this yum install gcc-c++ On Wed, Jul 3, 2013 at 1:54 PM, Daniel Kranich dan...@nusalaska.com wrote: Hello, I have been trying to compile GDAL on my system as I would like to enable BigTIFF support. I got through the ./compile just fine, however the make is giving me issues right off the start. Here is what my make command does: [root@daniel-fedora gdal-1.10.0]# make (cd port; make) make[1]: Entering directory `/home/Daniel/Downloads/gdal-** 1.10.0/port' /bin/sh /home/Daniel/Downloads/gdal-1.**10.0/libtool --mode=compile --tag=CXX g++ -DOGR_ENABLED -I/home/Daniel/Downloads/gdal-**1.10.0/port -I../frmts/zlib -DHAVE_LIBZ -c -o cpl_conv.lo cpl_conv.cpp libtool: compile: g++ -DOGR_ENABLED -I/home/Daniel/Downloads/gdal-**1.10.0/port -I../frmts/zlib -DHAVE_LIBZ -c cpl_conv.cpp -o .libs/cpl_conv.o /home/Daniel/Downloads/gdal-1.**10.0/libtool: line 1133: g++: command not found make[1]: *** [cpl_conv.lo] Error 1 make[1]: Leaving directory `/home/Daniel/Downloads/gdal-**1.10.0/port' make: *** [port-target] Error 2 Im not even sure where to start looking. Thanks -Daniel __**_ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/**mailman/listinfo/gdal-devhttp://lists.osgeo.org/mailman/listinfo/gdal-dev ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] Accessing to GRIB bands
you can always convert the files to netcdf using the command-line tool cdo (climate data operators), although that would increase processing time and disk space. https://code.zmaw.de/projects/cdo Etienne On Mon, Jun 17, 2013 at 8:16 PM, Paolo Corti pco...@gmail.com wrote: On Tue, Jun 18, 2013 at 12:46 AM, Even Rouault even.roua...@mines-paris.org wrote: From gdal_translate man page: -b band: Select an input band band for output. Bands are numbered from 1. Multiple -b switches may be used to select a set of input bands to write to the output file, or to reorder bands. Starting with GDAL 1.8.0, band can also be set to 'mask,1' (or just 'mask') to mean the mask band of the first band of the input dataset. Hey Even thanks for the reply. I am well aware of the -b option, the problem here is that each raster has not the same bands order. Therefore, as I had to make some work on just one single band representing the same variable in all of the datasets, I ended up to write a script, iterate the datasets, then iterate the bands, read the metadata, and when found the correct band ran the process on it. thanks a lot in any case cheers p -- Paolo Corti Geospatial software developer web: http://www.paolocorti.net twitter: @capooti skype: capooti ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] gdal_translate file naming for sds and HDF4 files
I have created a ticket and a patch for this problem http://trac.osgeo.org/gdal/ticket/5119 On Fri, May 31, 2013 at 3:28 PM, Etienne Tourigny etourigny@gmail.comwrote: On Fri, May 31, 2013 at 1:04 PM, Jonathan Greenberg j...@illinois.eduwrote: Thanks -- I did notice it was an earlier version (switching to the latest OSGeo4W), I apologize for not checking the latest, but as Etienne confirmed, the bug seems to be there in the latest as well. The HDF files from various sources are so odd when it comes to naming layers, that this SDS argument is really helpful (if it would name the files correctly). Speaking of -sds -- perhaps I can request that gdal_translate allow for a layer # flag rather than the full layer name for subdataset files? Is this relatively easy to implement (since gdalinfo seems to already have the capability)? that depends on the driver actually... for hdf4 files normal syntax uses number, whereas the netcdf driver uses variable name (which is more informative). Changing this behaviour involves tweaking the driver code, not gdal_translate. e.g. SUBDATASET_1_NAME=HDF4_SDS:MODIS_L1B:GSUB1.A2001124.0855.003.200219309451.hdf:0 SUBDATASET_1_NAME=NETCDF:sst.nc:lon_bnds Now perhaps you mean to select a single dataset by number, without the full SUBDATASET_X_NAME syntax ? such as gdalinfo -subdatset 1 file.hdf which would be equivalent to gdalinfo HDF4_SDS:file.hdf:0 that would be marginally handy, but not much as the rest of gdal uses the driver:file:subdataset syntax. --j On Thu, May 30, 2013 at 3:50 PM, Etienne Tourigny etourigny@gmail.com wrote: On Thu, May 30, 2013 at 4:46 PM, Jukka Rahkonen jukka.rahko...@mmmtike.fi wrote: Jonathan Greenberg jgrn at illinois.edu writes: Folks: I'm using a Windows install of GDAL 1.7.0 (FWTools) -- when I run: gdal_translate -sds -of GTiff [some.hdf] myoutput The output filenames do not have the .tif extension. If I set the output file to myoutput.tif, it appends the layer numbers to the end of the file, e.g. myoutput.tif1. Any tricks to getting the naming working properly (myoutput1.tif would be what I'd expect)? Hi, First, acquire more fresh GDAL, current version is 1.10. Use OSGeo4W or download Windows binaries from http://gisinternals.com/sdk/ actually the same thing happens with gdal-1.10. I will prepare a bug report and fix. Then, perhaps you should use gdalinfo first for seeing the exact names of layers as GDAL understands them and use such a layer name as input instead of [some.hdf]. Read usage examples from http://www.gdal.org/frmt_hdf4.html the idea of using the -sds argument is to generate an output file for each subdataset, without having to search for subdataset names -Jukka Rahkonen- ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev -- Jonathan A. Greenberg, PhD Assistant Professor Global Environmental Analysis and Remote Sensing (GEARS) Laboratory Department of Geography and Geographic Information Science University of Illinois at Urbana-Champaign 607 South Mathews Avenue, MC 150 Urbana, IL 61801 Phone: 217-300-1924 http://www.geog.illinois.edu/~jgrn/ AIM: jgrn307, MSN: jgrn...@hotmail.com, Gchat: jgrn307, Skype: jgrn3007 ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] Aggregate bands
that should be a simple as gdal_translate -of AAIGrid in.nc out.asc On Thu, Jun 13, 2013 at 8:32 AM, Paul Meems bontepaar...@gmail.com wrote: Hi list, I have a netCDF file with 24 bands. I want to create a new (ASCII) grid with the bands aggregated. Is this possible using the gdal tools or with an existing python script? Thanks, Paul ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] gdal_merge does not mosaic
On Tue, Jun 4, 2013 at 6:59 AM, adi_khan adiba.niz...@gmail.com wrote: Hi All, Could you please suggest any open source library/API/ command line tools (on Linux) that can mosaic images of the kind I posted earlier (with different paramters etc..)? Also if I am to use gdal_merge then I will have to reproject all input images to one projcs and then try mosaicing ? why don't you try the command that was suggested earlier? gdalwarp -of HFA -t_srs [some_definition_here] -srsnodata 0 0 0 img1.img img2.img mosaic.img -- View this message in context: http://osgeo-org.1560.x6.nabble.com/gdal-merge-does-not-mosaic-tp5057095p5057704.html Sent from the GDAL - Dev mailing list archive at Nabble.com. ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] gdal_merge does not mosaic
Update to gdal 1.9 and try again. On Fri, May 31, 2013 at 2:36 AM, adi_khan adiba.niz...@gmail.com wrote: I am using gdal 1.4.5 and wish to use gdal_merge to mosaic *.img images. The command runs successfully (no error is reported) but the output file contains the last input image and not mosaiced output. I use openEV to see the output file. This is how is used the command : gdal_merge.py -of HFA -o mosaic.img img1.img img2.img img3.img -- View this message in context: http://osgeo-org.1560.x6.nabble.com/gdal-merge-does-not-mosaic-tp5057095.html Sent from the GDAL - Dev mailing list archive at Nabble.com. ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] gdal_merge does not mosaic
On Fri, May 31, 2013 at 7:47 PM, Jean-Claude Repetto jrepe...@free.frwrote: On 31/05/2013 19:48, adi_khan wrote: Thanks for your reply frank. I understand that the GDAL I am using is ancient, but considering 'upgrading to other version' not an option could you tell me if there's a way I can mosaic using 1.4.5 only? Another option is to use gdalwarp instead of gdal_merge.py : gdalwarp -of HFA img1.img img2.img img3.img mosaic.img I hadn't even noticed that gdal_merge was being used. gdalwarp is far better than gdal_merge.py IMHO __**_ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/**mailman/listinfo/gdal-devhttp://lists.osgeo.org/mailman/listinfo/gdal-dev ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] gdal_translate file naming for sds and HDF4 files
On Thu, May 30, 2013 at 4:46 PM, Jukka Rahkonen jukka.rahko...@mmmtike.fiwrote: Jonathan Greenberg jgrn at illinois.edu writes: Folks: I'm using a Windows install of GDAL 1.7.0 (FWTools) -- when I run: gdal_translate -sds -of GTiff [some.hdf] myoutput The output filenames do not have the .tif extension. If I set the output file to myoutput.tif, it appends the layer numbers to the end of the file, e.g. myoutput.tif1. Any tricks to getting the naming working properly (myoutput1.tif would be what I'd expect)? Hi, First, acquire more fresh GDAL, current version is 1.10. Use OSGeo4W or download Windows binaries from http://gisinternals.com/sdk/ actually the same thing happens with gdal-1.10. I will prepare a bug report and fix. Then, perhaps you should use gdalinfo first for seeing the exact names of layers as GDAL understands them and use such a layer name as input instead of [some.hdf]. Read usage examples from http://www.gdal.org/frmt_hdf4.html the idea of using the -sds argument is to generate an output file for each subdataset, without having to search for subdataset names -Jukka Rahkonen- ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] vsipreload: enabling VSI Virtual File API for regular I/O
On Tue, May 28, 2013 at 11:24 AM, Howard Butler hobu@gmail.com wrote: On May 26, 2013, at 9:46 AM, Even Rouault even.roua...@mines-paris.org wrote: This file is the source code for a standalone shared library that can be LD_PRELOAD'ed as an overload of libc to enable VSI Virtual FILE API to be used with binaries using regular libc I/O API. This can work with all VSI Large File API filesystems : /vsizip/ , /vsitar/, /vsisubfile/ , etc... Cool! Way cool, Even! I'll test it when I have the time. Next question, if you'll pardon my ignorance, why would we ever want to do this? What does the VSI preload provide? You can open files over ftp/http connexions and files that are compressed inside zip/tgz/gz archives. This was not previously possible with certain drivers (hdf4, netcdf, etc.) Howard ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] vsipreload: enabling VSI Virtual File API for regular I/O
On Sun, May 26, 2013 at 11:46 AM, Even Rouault even.roua...@mines-paris.org wrote: Hi, I've just commited in trunk a new file port/vsipreload.cpp. This file is the source code for a standalone shared library that can be LD_PRELOAD'ed as an overload of libc to enable VSI Virtual FILE API to be used with binaries using regular libc I/O API. WARNING: Linux glibc ONLY. Might work with some adaptations (mainly around 64bit symbols) on other Unix systems Compile: g++ -Wall -fPIC port/vsipreload.cpp -shared -o vsipreload.so -Iport \ -L. -L.libs -lgdal Examples : LD_PRELOAD=vsipreload.so gdalinfo \ /vsicurl/http://download.osgeo.org/gdal/data/ecw/spif83.ecw Even, I have built vsipreload with a fresh build of svn trunk, but I get the following error: $ g++ -Wall -fPIC port/vsipreload.cpp -shared -o vsipreload.so -Iport -L. -L.libs -lgdal $ LD_PRELOAD=vsipreload.so gdalinfo /vsicurl/ http://download.osgeo.org/gdal/data/ecw/spif83.ecw ERROR: ld.so: object 'vsipreload.so' from LD_PRELOAD cannot be preloaded: ignored. ERROR 4: `/vsicurl/http://download.osgeo.org/gdal/data/ecw/spif83.ecw' not recognised as a supported file format. gdalinfo failed - unable to open '/vsicurl/ http://download.osgeo.org/gdal/data/ecw/spif83.ecw'. Is there a way to find out why it cannot be preloaded? Perhaps the compilation options should be tweaked? Thanks, Etienne My system is Linux Mint 13 ubuntu (quantal), gcc version 4.7.2, 64bits, 3.5.0-30-generic kernel LD_PRELOAD=vsipreload.so gdalinfo \ 'HDF4_EOS:EOS_GRID:/vsicurl/ http://download.osgeo.org/gdal/data/hdf4/MOD09Q1G_EVI.A2006233.h07v03.005.2008338190308.hdf :MODIS_NACP_EVI:MODIS_EVI' LD_PRELOAD=vsipreload.so ogrinfo \ /vsicurl/ http://svn.osgeo.org/gdal/trunk/autotest/ogr/data/testavc -ro LD_PRELOAD=vsipreload.so ogrinfo -ro \ /vsizip//vsicurl/ http://download.osgeo.org/gdal/1.10.0/gdalautotest-1.10.0.zip/gdalautotest-1.10.0/ogr/data/testavc It can work even with non GDAL binaries : LD_PRELOAD=vsipreload.so h5dump -d /x \ /vsicurl/ http://download.osgeo.org/gdal/data/netcdf/utm-big-chunks.nc LD_PRELOAD=vsipreload.so sqlite3 \ /vsicurl/http://download.osgeo.org/gdal/data/sqlite3/polygon.db \ select * from polygon limit 10 This can work with all VSI Large File API filesystems : /vsizip/ , /vsitar/, /vsisubfile/ , etc... This is still a bit experimental in the sense that only the most common glibc I/O API have been overloaded. If exotic ones are used with /vsi files, crashes are likely. -- Geospatial professional services http://even.rouault.free.fr/services.html ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] vsipreload: enabling VSI Virtual File API for regular I/O
On Tue, May 28, 2013 at 6:57 PM, Etienne Tourigny etourigny@gmail.comwrote: On Sun, May 26, 2013 at 11:46 AM, Even Rouault even.roua...@mines-paris.org wrote: Hi, I've just commited in trunk a new file port/vsipreload.cpp. This file is the source code for a standalone shared library that can be LD_PRELOAD'ed as an overload of libc to enable VSI Virtual FILE API to be used with binaries using regular libc I/O API. WARNING: Linux glibc ONLY. Might work with some adaptations (mainly around 64bit symbols) on other Unix systems Compile: g++ -Wall -fPIC port/vsipreload.cpp -shared -o vsipreload.so -Iport \ -L. -L.libs -lgdal Examples : LD_PRELOAD=vsipreload.so gdalinfo \ /vsicurl/http://download.osgeo.org/gdal/data/ecw/spif83.ecw Even, I have built vsipreload with a fresh build of svn trunk, but I get the following error: $ g++ -Wall -fPIC port/vsipreload.cpp -shared -o vsipreload.so -Iport -L. -L.libs -lgdal $ LD_PRELOAD=vsipreload.so gdalinfo /vsicurl/ http://download.osgeo.org/gdal/data/ecw/spif83.ecw ERROR: ld.so: object 'vsipreload.so' from LD_PRELOAD cannot be preloaded: ignored. ERROR 4: `/vsicurl/http://download.osgeo.org/gdal/data/ecw/spif83.ecw' not recognised as a supported file format. gdalinfo failed - unable to open '/vsicurl/ http://download.osgeo.org/gdal/data/ecw/spif83.ecw'. Is there a way to find out why it cannot be preloaded? Perhaps the compilation options should be tweaked? I have managed to get it working buy using LD_PRELOAD=./vsipreload.so e.g. LD_PRELOAD=./vsipreload.so gdalinfo /vsicurl/ http://download.osgeo.org/gdal/data/netcdf/utm-big-chunks.nc It seems that setting LD_PRELOAD outside of the command works also, so this mechanism would be used transparently. But does it have a significant overhead and/or chances it might do damage if always set? e.g. export LD_PRELOAD=./vsipreload.so gdalinfo /vsicurl/ http://download.osgeo.org/gdal/data/netcdf/utm-big-chunks.nc Is there any way this could be integrated into gdal to transparently support vsifile for all drivers, without setting LD_PRELOAD? In this way this feature would be available to all gdal commands or even other apps that use gdal. For example, /vsizip/ is used in QGIS, but it does not work with certain files (e.g. netcdf ) inside archives. But if I use LD_PRELOAD it works for netcdf files! Neat stuff! e.g. LD_PRELOAD=./vsipreload.so qgis great work Etienne Thanks, Etienne My system is Linux Mint 13 ubuntu (quantal), gcc version 4.7.2, 64bits, 3.5.0-30-generic kernel LD_PRELOAD=vsipreload.so gdalinfo \ 'HDF4_EOS:EOS_GRID:/vsicurl/ http://download.osgeo.org/gdal/data/hdf4/MOD09Q1G_EVI.A2006233.h07v03.005.2008338190308.hdf :MODIS_NACP_EVI:MODIS_EVI' LD_PRELOAD=vsipreload.so ogrinfo \ /vsicurl/ http://svn.osgeo.org/gdal/trunk/autotest/ogr/data/testavc -ro LD_PRELOAD=vsipreload.so ogrinfo -ro \ /vsizip//vsicurl/ http://download.osgeo.org/gdal/1.10.0/gdalautotest-1.10.0.zip/gdalautotest-1.10.0/ogr/data/testavc It can work even with non GDAL binaries : LD_PRELOAD=vsipreload.so h5dump -d /x \ /vsicurl/ http://download.osgeo.org/gdal/data/netcdf/utm-big-chunks.nc LD_PRELOAD=vsipreload.so sqlite3 \ /vsicurl/http://download.osgeo.org/gdal/data/sqlite3/polygon.db\ select * from polygon limit 10 This can work with all VSI Large File API filesystems : /vsizip/ , /vsitar/, /vsisubfile/ , etc... This is still a bit experimental in the sense that only the most common glibc I/O API have been overloaded. If exotic ones are used with /vsi files, crashes are likely. -- Geospatial professional services http://even.rouault.free.fr/services.html ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] vsipreload: enabling VSI Virtual File API for regular I/O
On Tue, May 28, 2013 at 7:35 PM, Even Rouault even.roua...@mines-paris.orgwrote: I have managed to get it working buy using LD_PRELOAD=./vsipreload.so Yes for some strange reason, some systems require the ./ and some not... My old Ubuntu 10.04 does no, but Travis on Ubuntu 12.04 does. Perhaps some security measure that has been added in LD_PRELOAD mechanism. I've just adjusted the examples in the .cpp header to add ./ e.g. LD_PRELOAD=./vsipreload.so gdalinfo /vsicurl/ http://download.osgeo.org/gdal/data/netcdf/utm-big-chunks.nc It seems that setting LD_PRELOAD outside of the command works also, so this mechanism would be used transparently. But does it have a significant overhead and/or chances it might do damage if always set? This should have no impact for I/O on regular files, except for a hopefully small performance hit to check if the FILE* is a regular one (and redirect to the glibc function) or one that is a VSILFILE* instead (and redirect to VSI implementation). On /vsi files, crashes are theoretically possible if a glibc function using the fake FILE* is used, but not overloaded in the .so. e.g. export LD_PRELOAD=./vsipreload.so gdalinfo /vsicurl/ http://download.osgeo.org/gdal/data/netcdf/utm-big-chunks.nc Is there any way this could be integrated into gdal to transparently support vsifile for all drivers, without setting LD_PRELOAD? None that I'm aware of. LD_PRELOAD is really something that must be specified before process launching. When GDAL code is reached, it is too late (and those limitations are really welcome for obvious security reasons). It would probably best (and safer for the time being) to use aliases for commonly-used apps that would benefit from this (gdal*, qgis, etc) alias gdalinfo='LD_PRELOAD=/usr/local/lib/vsipreload.so gdalinfo' alias gdalwarp='LD_PRELOAD=/usr/local/lib/vsipreload.so gdalwarp' alias qgis='LD_PRELOAD='usr/local/lib/vsipreload.so qgis' In this way this feature would be available to all gdal commands or even other apps that use gdal. For example, /vsizip/ is used in QGIS, but it does not work with certain files (e.g. netcdf ) inside archives. But if I use LD_PRELOAD it works for netcdf files! Neat stuff! e.g. LD_PRELOAD=./vsipreload.so qgis As I mentioned in another post in that thread, the use of a fuse filesystem could make it less hackish, although you would need permissions to mount on /vsizip , /vsicurl etc which might be problematic if you are not a good friend with the system administrator. You don't need to be root to mount a fuse filesystem, but you must have write permissions on the mount point I believe. it seems a little more involved and less flexible great work Etienne -- Geospatial professional services http://even.rouault.free.fr/services.html ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] NetCDF grid mapping read issue
On Wed, May 22, 2013 at 2:10 PM, Andreas Oxenstierna a...@t-kartor.se wrote: Dear list, GRID cannot interpret (tested with 1.9 and 1.10) the coordinate systems in two different NetCDF files storing oceanographic/meteorological data. But it can interpret the coordinates in a third NetCDF file. PANOPLY, as available from http://www.giss.nasa.gov/**tools/panoply/http://www.giss.nasa.gov/tools/panoply/, can correctly display all files in lat-lon (4326). NetCDF file where GDAL can interpret the coordinates: https://dl.dropboxusercontent.**com/u/13880880/metoffice_** foam1_orca025_GLO_ASLV_**b20120923_dm20120921.nchttps://dl.dropboxusercontent.com/u/13880880/metoffice_foam1_orca025_GLO_ASLV_b20120923_dm20120921.nc (CF convention is 1.0) NetCDF files where GDAL cannot interpret the coordinates: https://dl.dropboxusercontent.**com/u/13880880/metoffice_** foam1_orca025_NTV_ASLV_**b20110921_dm20110921.nchttps://dl.dropboxusercontent.com/u/13880880/metoffice_foam1_orca025_NTV_ASLV_b20110921_dm20110921.nc (CF convention is 1.0) http://thredds.met.no/thredds/**fileServer/metno/proff4km/** default/files/proffDefault4km_**2013-04-25_00.nchttp://thredds.met.no/thredds/fileServer/metno/proff4km/default/files/proffDefault4km_2013-04-25_00.nc(be warned: 1.2 Gb)(CF convention is 1.4) We have got this information from the Norwegian experts: For PROFF data, the rotated lat/lon coordinates are defined so that each latitude and each longitude (and therefore each data point) correspond to elements in the rlat/rlon vectors. rotated pole is not supported, sorry. If the driver supported the netcdf-cf coordinates attribute (see Netcdf CF 1.6 section 5.2) these 2 problems would be solved... There is a mention of this in the netcdf improvement wiki at http://trac.osgeo.org/gdal/wiki/NetCDF_Improvements Please add this information to the following ticket http://trac.osgeo.org/gdal/ticket/4285 regards, Etienne The GeoTransform array mentioned in http://www.gdal.org/frmt_**netcdf.htmlhttp://www.gdal.org/frmt_netcdf.htmlseems not to be used for these two files. -- Best Regards Andreas Oxenstierna T-Kartan Produkt AB mobile: +46 733 206831 mailto: a...@t-kartor.se http://www.t-kartor.com __**_ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/**mailman/listinfo/gdal-devhttp://lists.osgeo.org/mailman/listinfo/gdal-dev ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] multiple access to a file
On Tue, May 21, 2013 at 7:30 AM, Jose Gomez-Dans jgomezd...@gmail.comwrote: Hi Frank, On 21 May 2013 00:02, Frank Warmerdam warmer...@pobox.com wrote: If you write a small part of an image, GDAL is likely to read a larger area, update that and write it back. If other programs happen to try and update the same larger area at the same time some data is likely to be lost. Thanks for your reply. However, locking the output files would still work? Provided the cache has been flushed between the open/close dataset calls, it should be fine? I guess it would work, but you probably won't get much speedup this way. Like Frank says, not sure it would be so trivial as you would have to implement something like MPI in the gdal I/O routines and take care that blocks are assigned to each MPI process. If you have many files to process, it might be easier to distribute the many files over many machines simultaneously, but write one file per machine only. cheers Etienne Thanks! Jose ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] problem with gdal_translate
Please look at the netcdf docs at [1] When there are various variables in a netcdf ffile, gdal treats them as subdatasets, you can only operate on one at a time. calling gdalinfo on the netcdf file will reveal the special names used to access each subdataset [1] http://www.gdal.org/frmt_netcdf.html On Mon, May 20, 2013 at 3:34 PM, Peng Fu fupengh...@gmail.com wrote: Hi, I used the gdal_translate to store the netcdf data into geotiff file, but it just gave me an error: Input file contains subdatasets. Please, select one of them for reading. command: gdal_translate -of GTiff -b 9 -a_srs EPSG:4326 BAND_02.nc /test.tif Does anyone know what is the problem? Thanks! Peng ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] problem with gdal_translate
On Mon, May 20, 2013 at 4:41 PM, Peng Fu fupengh...@gmail.com wrote: Thanks! I fixed the problem. But it seems that the resultant geotiff image doesn't have coordinate information. Do you know how to keep the coordinate information? that's probably because it doesn't have one... it is a lat/lon file right? did you add the -a_srs EPSG:4326 argument to gdal_translate? Perhaps you mean there is no geotransform? Or perhaps the coordinate information does not adhere to the CF Conventions [2], which gdal uses. In any case calling gdalinfo on the subdataset will show which coordinate information gdal gets from the netcdf file. Send me the output of both gdalinfo and 'ncdump -h yourfile.nc' [2] http://cf-pcmdi.llnl.gov/documents/cf-conventions/1.6/cf-conventions.html Peng On Mon, May 20, 2013 at 3:38 PM, Etienne Tourigny etourigny@gmail.com wrote: Please look at the netcdf docs at [1] When there are various variables in a netcdf ffile, gdal treats them as subdatasets, you can only operate on one at a time. calling gdalinfo on the netcdf file will reveal the special names used to access each subdataset [1] http://www.gdal.org/frmt_netcdf.html On Mon, May 20, 2013 at 3:34 PM, Peng Fu fupengh...@gmail.com wrote: Hi, I used the gdal_translate to store the netcdf data into geotiff file, but it just gave me an error: Input file contains subdatasets. Please, select one of them for reading. command: gdal_translate -of GTiff -b 9 -a_srs EPSG:4326 BAND_02.nc /test.tif Does anyone know what is the problem? Thanks! Peng ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] compile
On Sun, May 19, 2013 at 4:24 PM, Peng FU fupengh...@gmail.com wrote: Hi, ** ** Does anyone know how to build gdal in linux OS or windows OS? I want to include the netcdf library into the gdal? FYI some builds of gdal include netcdf support, such as ubuntugis-unstable PPA (for ubuntu and derivatives). Thanks! ** ** peng ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] gdalwarp does not apply source nodata value to destination
I have created a ticket and attached a patch to resolve this. http://trac.osgeo.org/gdal/ticket/5087 On Wed, May 15, 2013 at 11:12 AM, Etienne Tourigny etourigny@gmail.comwrote: I have just noticed that destination nodata value is not set when -dstnodata option is not used in gdalwarp. This can be problematic when warping a dataset with no data values, as the result can be different from expected, if -dstnodata is not set. gdalwarp does use source nodata pixels if -srcnodata is not used. I consider this as a bug (or at the very least a limitation), because one cannot warp a dataset in an automatic fashion and get expected output (with same nodata value as input). Would it be OK to copy them over if -dstnodata is not used, or would there be problems with doing this? Or perhaps a new argument -copynodata could be added if there is fear of disruption. cheers Etienne ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
[gdal-dev] gdalwarp does not apply source nodata value to destination
I have just noticed that destination nodata value is not set when -dstnodata option is not used in gdalwarp. This can be problematic when warping a dataset with no data values, as the result can be different from expected, if -dstnodata is not set. gdalwarp does use source nodata pixels if -srcnodata is not used. I consider this as a bug (or at the very least a limitation), because one cannot warp a dataset in an automatic fashion and get expected output (with same nodata value as input). Would it be OK to copy them over if -dstnodata is not used, or would there be problems with doing this? Or perhaps a new argument -copynodata could be added if there is fear of disruption. cheers Etienne ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] Raster Attribute Table RFC
On Tue, May 14, 2013 at 5:54 PM, Frank Warmerdam warmer...@pobox.comwrote: Sam, It looks good to me, though I'm dubious about the value of treating booleans as Byte instead of integer. everywhere else booleans are defined as integers like this int bSomeVar; Best regards, Frank On Mon, May 13, 2013 at 9:37 PM, Sam Gillingham gillingham@gmail.comwrote: Hi All, I propose a motion to get RFC 40 - Improving performance of Raster Attribute Table implementation for large tables adopted. This adds some new functionality for GDAL 2.0. Even and others have made suggestions and these have been incorporated into the RFC: http://trac.osgeo.org/gdal/wiki/rfc40_enhanced_rat_support Thanks, Sam. ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev -- ---+-- I set the clouds in motion - turn up | Frank Warmerdam, warmer...@pobox.com light and sound - activate the windows | http://pobox.com/~warmerdam and watch the world go round - Rush| Geospatial Software Developer ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] Compile GDAL/OGR with SOSI support
Whoever is responsible for the GDAL SOSI driver (at the github project linked in the wiki) should consider contributing their code to the main gdal codebase (dev version/trunk of course) Cheers Etienne On Thu, Apr 25, 2013 at 5:58 AM, SBL stefan.blumentr...@nina.no wrote: Done. See: http://trac.osgeo.org/gdal/wiki/SOSI http://trac.osgeo.org/gdal/wiki/SOSI Cheers Stefan -- View this message in context: http://osgeo-org.1560.x6.nabble.com/Compile-GDAL-OGR-with-SOSI-support-tp5045207p5049558.html Sent from the GDAL - Dev mailing list archive at Nabble.com. ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] MODIS reprojection
I use a similar command, without the -a_ullr argument e.g. gdal_translate -a_srs '+proj=sinu +R=6371007.181 +nadgrids=@null +wktext' input HDF subdataset output tiff file On Thu, Apr 18, 2013 at 6:10 AM, Rutger kass...@gmail.com wrote: Hey Laura, Since the MxD14.A2 products are already projected in the sinusoidal projection there is no need to warp using GCP's. Fitting a polynomial through some GCP's will always be less accurate then using the projection the data is already in. Somewhere in the metadata in the HDF there should be a boundingbox (corner coordinates, in sinusoidal projection) for the specific tile you are using, you can then assign these with the projection using gdal_translate. Once assigned, you can use gdalwarp in the same way as you already described. Youre gdal_translate command should look somewhat like this: gdal_translate -of VRT -a_srs +proj=sinu +lon_0=0 +x_0=0 +y_0=0 +a=6371007.181 +b=6371007.181 +units=m +no_defs -a_ullr 2223901.039333 -3335851.559000 3335851.559000 -4447802.078667 input HDF subdataset output vrt file You can get the correct subdataset reference by running gdalinfo on you HDF. The coordinates i used are for tile h20v12, so make sure you use the ones from your tile. I used a vrt as output which you can use as an input for gdal_translate, but its also fine to use Geotiff or anything else you prefer. Hope this helps. Regards, Rutger -- View this message in context: http://osgeo-org.1560.x6.nabble.com/MODIS-reprojection-tp5047700p5047887.html Sent from the GDAL - Dev mailing list archive at Nabble.com. ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] MODIS reprojection
On Thu, Apr 18, 2013 at 12:10 PM, Rutger kass...@gmail.com wrote: Hey, Etienne Tourigny-3 wrote I use a similar command, without the -a_ullr argument And does it then capture the correct corner coordinates? If i run your command it does assign the sinusoidal projection, but takes the pixel coordinates (eg 0-2048) as the corner coordinates instead of the sinusoidal ones. Actually, I used this with the level-3 tiled products such as MCD12Q1 or MCD45A1. Never tried with level-2 or swath products. Yes the corners are fine, I can convert a few of them to WGS84 and then stitch them together with gdalwarp. Etienne @Laura, Indeed we are talking about different products, sorry for the confusion. You mentioned in your first email the products MYD14.A2 and MOD14.A2. I thought you meant the MYD14 A2 product, but you attached a MYD14 product, which is different. Its obvious where the confusion comes from. :) These are all the fire products: MYD14 AquaThermal Anomalies FireSwath 1000m 5 min MOD14A2 Terra Thermal Anomalies FireTile1000m 8 day MYD14A2 AquaThermal Anomalies FireTile1000m 8 day MOD14A1 Terra Thermal Anomalies FireTile1000m Daily MYD14A1 AquaThermal Anomalies FireTile1000m Daily MOD14 Terra Thermal Anomalies FireSwath 1000m 5 min The MOD14 MYD14 products are indeed swath, and should be used together with the MOD03 with the same timestamp. I think the best way to process them is by using MRT Swath which you can download at: https://lpdaac.usgs.gov/tools MRT Swath is way beyond the scope of this mailing list, but feel free to send me a direct email about it if you have questions. I have used Pyresample as an alternative which works very well, but requires the use of Python: http://code.google.com/p/pyresample/ To end with an actual GDAL related solution, there is an option to warp an image using gdalwarp and so-called geolocation arrays. I can get it to work with the MODIS swaths, but i dont like the results at all. It is previously discussed at: http://osgeo-org.1560.x6.nabble.com/gdal-dev-reproject-python-numpy-binary-swath-lat-lon-td4978609.html The MODIS swath products arent equally gridded and still contain what is called the 'bow-tie' effect. (overlapping scans at high zenith angles). I dont know if the GDAL geoloc option is supposed to handle this properly. The image a posted earlier in the thread mentioned above clearly shows artifacts which i cannot explain or resolve. Regards, Rutger -- View this message in context: http://osgeo-org.1560.x6.nabble.com/MODIS-reprojection-tp5047700p5048019.html Sent from the GDAL - Dev mailing list archive at Nabble.com. ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] lib to get from GDAL/geotiff to PROJ4
You can have a look at the code used by the gdalsrsinfo , which basically
opens a GDAL/OGR datasource and outputs the proj4 (or wkt) string.
Assuming you use c/c++ of course...
http://trac.osgeo.org/gdal/browser/trunk/gdal/apps/gdalsrsinfo.cpp
Here is the relevant code (some declarations missing)
poGDALDS = (GDALDataset *) GDALOpen( pszInput, GA_ReadOnly );
if ( poGDALDS != NULL poGDALDS-GetProjectionRef( ) != NULL ) {
pszProjection = (char *) poGDALDS-GetProjectionRef( );
if( oSRS.importFromWkt( pszProjection ) == CE_None ) {
CPLDebug( gdalsrsinfo, got SRS from GDAL );
bGotSRS = TRUE;
}
GDALClose( (GDALDatasetH) poGDALDS );
if ( ! bGotSRS )
CPLDebug( gdalsrsinfo, did not open with GDAL );
}
oSRS.exportToProj4( pszOutput );
printf( \'%s\'\n, pszOutput );
On Wed, Apr 17, 2013 at 8:23 AM, wsvries wsvr...@xs4all.nl wrote:
Hi,
I am working on an application (adding terrain info to maps for 3D
rendering) that does some heavy projection calculations using the proj4
libs.
Input for now is geotiff.
I have Googled a lot, but I was not able to find any libs that can output
proj4 strings based on geotiff input.
I have seen (GDAL based) utilities that are able to do that but no libs to
use in my application.
Any hint is welcome.
Thanks.
__**_
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/**mailman/listinfo/gdal-devhttp://lists.osgeo.org/mailman/listinfo/gdal-dev
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] Feature request: gauss and other interpolations in gdalwarp
The patch works successfully - it was a build environment problem. On Mon, Apr 8, 2013 at 3:17 AM, John Twilley math...@gmail.com wrote: Thank you, Etienne, for writing this code. I greatly appreciate it! I updated my svn checkout, then applied your patch with no errors. I tried to warp my landcover files which use color tables (which is why mode is so important to me) and they did not work. Here is the output: jmt@nala:~/git/mathuin/TopoMC$ /home/jmt/svn/gdal/apps/.libs/gdalwarp --debug -multi -tr 6 6 -te 2087232 2395968 2104512 2410176 -r mode regions/Provincetown/Datasets/L1L.tif L1L-new.tif Copying color table from regions/Provincetown/Datasets/L1L.tif to new file. Creating output file that is 2880P x 2368L. Processing input file regions/Provincetown/Datasets/L1L.tif. Warning: Input file regions/Provincetown/Datasets/L1L.tif has a color table, which will likely lead to bad results when using a resampling method other than nearest neighbour. Converting the dataset prior to 24/32 bit is advised. ERROR 5: GDALWarpOptions.Validate() eResampleArg=6 is not a supported value. jmt@nala:~/git/mathuin/TopoMC$ Let me know if I can provide any additional information to help you out! Jack. -- mathuin at gmail dot com On Sun, Apr 7, 2013 at 1:19 PM, Etienne Tourigny etourigny@gmail.comwrote: I have implemented mode and average warping algorithms, based on those used in overview creation. http://trac.osgeo.org/gdal/ticket/5049 This code has been tested on a few datasets (see attachments) with all datatypes (Byte, Int, Float) but not all cases have been looked into (e.g. color tables, various projections). Any feedback and testing would be appreciated. Cheers Etienne On Wed, Mar 27, 2013 at 6:13 PM, John Twilley math...@gmail.com wrote: I'm interested in this feature request as well. Adding the mode resampling algorithm to gdalwarp would be very beneficial to my projects, right up there with being able to access the warp API from Python. Is this at all possible? Should I submit a feature request on Trac, or what? Just let me know! Jack. -- mathuin at gmail dot com On Mon, Feb 18, 2013 at 8:17 AM, Jan Hartmann j.l.h.hartm...@uva.nlwrote: Hi devs, Would it be possible to add gauss and other interpolations to gdalwarp? At the moment I georeference large scans to 2000*2000 tiles at the most detailed scale, and then create 2000*2000 tiles at resolutions of 2, 4 6 etc times the original scale, using gdaladdo and gauss or other interpolations. It would help immensely if I could do that directly with gdalwarp. Funding would probably no problem. The question is: can and should it be done? Jan ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] Feature request: gauss and other interpolations in gdalwarp
I have committed new warping methods average and mode to trunk, this will be part of gdal-1.10 On Tue, Apr 9, 2013 at 10:01 AM, Etienne Tourigny etourigny@gmail.comwrote: The patch works successfully - it was a build environment problem. On Mon, Apr 8, 2013 at 3:17 AM, John Twilley math...@gmail.com wrote: Thank you, Etienne, for writing this code. I greatly appreciate it! I updated my svn checkout, then applied your patch with no errors. I tried to warp my landcover files which use color tables (which is why mode is so important to me) and they did not work. Here is the output: jmt@nala:~/git/mathuin/TopoMC$ /home/jmt/svn/gdal/apps/.libs/gdalwarp --debug -multi -tr 6 6 -te 2087232 2395968 2104512 2410176 -r mode regions/Provincetown/Datasets/L1L.tif L1L-new.tif Copying color table from regions/Provincetown/Datasets/L1L.tif to new file. Creating output file that is 2880P x 2368L. Processing input file regions/Provincetown/Datasets/L1L.tif. Warning: Input file regions/Provincetown/Datasets/L1L.tif has a color table, which will likely lead to bad results when using a resampling method other than nearest neighbour. Converting the dataset prior to 24/32 bit is advised. ERROR 5: GDALWarpOptions.Validate() eResampleArg=6 is not a supported value. jmt@nala:~/git/mathuin/TopoMC$ Let me know if I can provide any additional information to help you out! Jack. -- mathuin at gmail dot com On Sun, Apr 7, 2013 at 1:19 PM, Etienne Tourigny etourigny@gmail.com wrote: I have implemented mode and average warping algorithms, based on those used in overview creation. http://trac.osgeo.org/gdal/ticket/5049 This code has been tested on a few datasets (see attachments) with all datatypes (Byte, Int, Float) but not all cases have been looked into (e.g. color tables, various projections). Any feedback and testing would be appreciated. Cheers Etienne On Wed, Mar 27, 2013 at 6:13 PM, John Twilley math...@gmail.com wrote: I'm interested in this feature request as well. Adding the mode resampling algorithm to gdalwarp would be very beneficial to my projects, right up there with being able to access the warp API from Python. Is this at all possible? Should I submit a feature request on Trac, or what? Just let me know! Jack. -- mathuin at gmail dot com On Mon, Feb 18, 2013 at 8:17 AM, Jan Hartmann j.l.h.hartm...@uva.nlwrote: Hi devs, Would it be possible to add gauss and other interpolations to gdalwarp? At the moment I georeference large scans to 2000*2000 tiles at the most detailed scale, and then create 2000*2000 tiles at resolutions of 2, 4 6 etc times the original scale, using gdaladdo and gauss or other interpolations. It would help immensely if I could do that directly with gdalwarp. Funding would probably no problem. The question is: can and should it be done? Jan ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] Feature request: gauss and other interpolations in gdalwarp
Hi Even, Thanks for your input On Tue, Apr 9, 2013 at 3:34 PM, Even Rouault even.roua...@mines-paris.orgwrote: Le mardi 09 avril 2013 19:06:28, Etienne Tourigny a écrit : I have committed new warping methods average and mode to trunk, this will be part of gdal-1.10 Hi Etienne, It would be good if you could extend the autotest suite to add tests for those new warping methods. For that, you can likely take inspiration from the first tests of autotest/warp/warp.py. Reference images based on utmsmall.tif reference image is a bit big, but you can likely start from a smaller source image like byte.tif instead that will produce reference images of reasonable size to be put in svn. I will do that, thanks Regarding nAlgo == 2 (mode with foating point data), the allocations of pafVals and panSums have the potential to fail if warping is done on a large image whose floating point values are rarely identical. So I think that VSIRealloc shoud be used instead with a test on the result to fail properly. I'm also a bit doubtfull about the practical usefulness of this case on real data. There might also be a performance issue due to the loop //Check array for existing entry that is at the most inner level of the algorithm. Unless you have a practical use case, my personnal opinion would be to refuse to run mode resampling on floating point data, or implement it with a tolerance parameter to avoid strict floating point comparison (if that makes sense), but even that wouldn't guarantee reasonable performance and memory consumption. The logic was copied from existing code for overviews... It is rather inefficient when dealing with floating-point or even integer (16/32 bits). You are right that it is not really practical, but I decided to go ahead and implement it anyway. Mostly I implemented it to deal with 16 and 32 bit integers, and was lazy to implement an integer-only solution. So here is my suggestion : modify this to use 32-bit integers instead, and refuse to run with floating-point data. The same could be done to overviews, although that *might* cause some regressions (although as you point out, mode resampling doesn't make sense for floating-point data). Or another solution is to implement a new integer algorithm, and leave the floating-point one around but print a warning. As far as mode resampling is concerned, it would likely run correctly for images with color tables, so I believe that the warning in gdalwarp.cpp at line 1048 could be shut down in that case too (in addition to nearest resampling) I have tested with a color table, and mode resampling does work. I forgot to update gdalwarp for this case, thanks. Best regards, Even ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] Feature request: gauss and other interpolations in gdalwarp
On Tue, Apr 9, 2013 at 6:50 PM, Even Rouault
even.roua...@mines-paris.orgwrote:
Le mardi 09 avril 2013 20:34:40, Even Rouault a écrit :
Le mardi 09 avril 2013 19:06:28, Etienne Tourigny a écrit :
I have committed new warping methods average and mode to trunk, this
will
be part of gdal-1.10
Hi Etienne,
It would be good if you could extend the autotest suite to add tests for
those new warping methods. For that, you can likely take inspiration from
the first tests of autotest/warp/warp.py. Reference images based on
utmsmall.tif reference image is a bit big, but you can likely start from
a
smaller source image like byte.tif instead that will produce reference
images of reasonable size to be put in svn.
Regarding nAlgo == 2 (mode with foating point data), the allocations of
pafVals and panSums have the potential to fail if warping is done on a
large image whose floating point values are rarely identical. So I think
that VSIRealloc shoud be used instead with a test on the result to fail
properly. I'm also a bit doubtfull about the practical usefulness of this
case on real data. There might also be a performance issue due to the
loop
//Check array for existing entry that is at the most inner level of the
algorithm.
I stand corrected on the above comment about big memory consumption. The
size
of the array is limited to the number of source pixels needed to compute a
target pixel, so unless you do extreme subsampling, that should be OK.
yes
I've just noticed however that pafVals and panSums aren't free'd, so
there's a
memory leak currently. And the CPLRealloc() are a bit weird as currently
coded
:
int nMaxNumPx = 0;
float* pafVals = NULL;
int* panSums = NULL;
if (nNumPx nMaxNumPx)
{
pafVals = (float*) CPLRealloc(pafVals, nNumPx *
sizeof(float));
panSums = (int*) CPLRealloc(panSums, nNumPx *
sizeof(int));
nMaxNumPx = nNumPx;
}
The test is always true, so CPLMalloc() would be clearer. But I think there
was a will to move nMaxNumPx, pafVals, panSums before the top loops. So
that
should likely be done.
I thought it is weird also, but again copied over from overview code.
I thought about running valgrind, but then forgot. I will take your
suggestions into consideration, thanks!
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] Feature request: gauss and other interpolations in gdalwarp
Please upload your file to the ticket (or just a small subset if the file is sensitive, or a way to generate a similar file, or send it to me). It is probably because of the color table which I have not tested for. Etienne On Mon, Apr 8, 2013 at 3:17 AM, John Twilley math...@gmail.com wrote: Thank you, Etienne, for writing this code. I greatly appreciate it! I updated my svn checkout, then applied your patch with no errors. I tried to warp my landcover files which use color tables (which is why mode is so important to me) and they did not work. Here is the output: jmt@nala:~/git/mathuin/TopoMC$ /home/jmt/svn/gdal/apps/.libs/gdalwarp --debug -multi -tr 6 6 -te 2087232 2395968 2104512 2410176 -r mode regions/Provincetown/Datasets/L1L.tif L1L-new.tif Copying color table from regions/Provincetown/Datasets/L1L.tif to new file. Creating output file that is 2880P x 2368L. Processing input file regions/Provincetown/Datasets/L1L.tif. Warning: Input file regions/Provincetown/Datasets/L1L.tif has a color table, which will likely lead to bad results when using a resampling method other than nearest neighbour. Converting the dataset prior to 24/32 bit is advised. ERROR 5: GDALWarpOptions.Validate() eResampleArg=6 is not a supported value. jmt@nala:~/git/mathuin/TopoMC$ Let me know if I can provide any additional information to help you out! Jack. -- mathuin at gmail dot com On Sun, Apr 7, 2013 at 1:19 PM, Etienne Tourigny etourigny@gmail.comwrote: I have implemented mode and average warping algorithms, based on those used in overview creation. http://trac.osgeo.org/gdal/ticket/5049 This code has been tested on a few datasets (see attachments) with all datatypes (Byte, Int, Float) but not all cases have been looked into (e.g. color tables, various projections). Any feedback and testing would be appreciated. Cheers Etienne On Wed, Mar 27, 2013 at 6:13 PM, John Twilley math...@gmail.com wrote: I'm interested in this feature request as well. Adding the mode resampling algorithm to gdalwarp would be very beneficial to my projects, right up there with being able to access the warp API from Python. Is this at all possible? Should I submit a feature request on Trac, or what? Just let me know! Jack. -- mathuin at gmail dot com On Mon, Feb 18, 2013 at 8:17 AM, Jan Hartmann j.l.h.hartm...@uva.nlwrote: Hi devs, Would it be possible to add gauss and other interpolations to gdalwarp? At the moment I georeference large scans to 2000*2000 tiles at the most detailed scale, and then create 2000*2000 tiles at resolutions of 2, 4 6 etc times the original scale, using gdaladdo and gauss or other interpolations. It would help immensely if I could do that directly with gdalwarp. Funding would probably no problem. The question is: can and should it be done? Jan ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] Feature request: gauss and other interpolations in gdalwarp
I have implemented mode and average warping algorithms, based on those used in overview creation. http://trac.osgeo.org/gdal/ticket/5049 This code has been tested on a few datasets (see attachments) with all datatypes (Byte, Int, Float) but not all cases have been looked into (e.g. color tables, various projections). Any feedback and testing would be appreciated. Cheers Etienne On Wed, Mar 27, 2013 at 6:13 PM, John Twilley math...@gmail.com wrote: I'm interested in this feature request as well. Adding the mode resampling algorithm to gdalwarp would be very beneficial to my projects, right up there with being able to access the warp API from Python. Is this at all possible? Should I submit a feature request on Trac, or what? Just let me know! Jack. -- mathuin at gmail dot com On Mon, Feb 18, 2013 at 8:17 AM, Jan Hartmann j.l.h.hartm...@uva.nlwrote: Hi devs, Would it be possible to add gauss and other interpolations to gdalwarp? At the moment I georeference large scans to 2000*2000 tiles at the most detailed scale, and then create 2000*2000 tiles at resolutions of 2, 4 6 etc times the original scale, using gdaladdo and gauss or other interpolations. It would help immensely if I could do that directly with gdalwarp. Funding would probably no problem. The question is: can and should it be done? Jan ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] Multiple extensions in GDAL_DMD_EXTENSION ?
On Tue, Feb 19, 2013 at 12:59 PM, Frank Warmerdam warmer...@pobox.com wrote: On Tue, Feb 19, 2013 at 9:39 AM, Etienne Tourigny etourigny@gmail.com wrote: It would be nice to be able to specify many extensions in a comma-separated list such as nc,cdf,nc4. It can probably be done (metadata is just a string), but isn't standard. The Gdal driver tutorial states: GDAL_DMD_EXTENSION: The extension used for files of this type. If more than one pick the primary extension, or none at all. (optional) It could be beneficial to extend this definition for multiple extensions separated by comma. Etienne, The challenge is to do this without breaking lots of applications that expect it to be a single extension. If we were to pursue this, I'd almost prefer to add GDAL_DMD_EXTENSION_LIST as a list of extensions that relate to this driver, and GDAL_DMD_EXTENSION would be the primary or preferred extension. But I'm still not all that keen. I see the GDAL_DMD_EXTENSION as existing primarily so that applications can prepare a reasonable default filename when creating a file. For this having a list of extensions is not helpful. People *usually* want a list of extensions so they can automatically categorize the format of files on disk or to filter to only show extensions they think GDAL supports. I find this practice abhorrent! GDAL is based on the idea that file formats are discovered by inspecting file contents rather than based on extension and I don't want to make it too easy for applications to ignore this principle and revert to extensions. Frank, thanks for you thorough response. I agree that GDAL's detection is rather efficient, but sometimes it can be relatively slow when scanning many files in a GUI. This is admittedly a worst-case scenario, but is the reason why I chose to optionally scan files by extension (rather than content) in QGis - this can be enabled for specific directories. One other use that extensions are used for is to make it easy for a user to filter down files in a file browser to just one format. I can't think of any time I've found this useful. But with GDAL there are so many formats that these list of formats filter dohickyies are usually completely unusable. Applications tend to have these, with ad-hoc rules for formats which use various extensions (e.g. the QGis raster open dialog). This would be simpler and more unified if extensions are reported by GDAL. In short, I'd support a new GDAL_DMD_EXTENSION_LIST that supplements existing GDAL_DMD_EXTENSION . I could commit such a feature in the netcdf driver and gdalinfo output and update the docs if you agree. Cheers Etienne Best regards, -- ---+-- I set the clouds in motion - turn up | Frank Warmerdam, warmer...@pobox.com light and sound - activate the windows | http://pobox.com/~warmerdam and watch the world go round - Rush| Geospatial Software Developer ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
