[gdal-dev] windows install - best practices? install from source, or from precompiled binaries?
I'm working reasonably successfully with GDAL as a learner, in command line mode using warp and translate, and then from within C++ in Visual Studio. I had first used GDAL from Python a year or two ago, before resuming pretty intensely via C++ recently. I had a problem with my PC, so I went to a new machine in the last few days. Oh my, how little nitty things can challenge, to restabilize, particularly when I don't have complete understanding of what I am doing. Things are finally working, but I didn't get there in a straight line at all, and I might have some redundant stuff on my new machine, so I'd like some advice please. Specifically, I think I installed twice unnecessarily, as follows: 1. I did the install from source using the https://trac.osgeo.org/gdal/wiki/DownloadSource page, then followed this youtube video for every detail: https://www.youtube.com/watch?v=Yf8rYOfvZjY. With a couple of tweaks, I ended up with all the GDAL folders and files in one directory, c:\gdal. 2. I did a much simpler install of .msi precompiled binaries using the Generic Installer for the Core components from: http://www.gisinternals.com/. I ended up with all the x64 gdal libraries in C:\Program Files\GDAL. This youtube was helpful: https://www.youtube.com/watch?v=hQt9tmfl-x8 Things would not work...I got some post-compile execution errors about not being able to find gdal203.dll. I found this youtube video ( https://www.youtube.com/watch?v=yKTeNVvF4gM ) that guided me to fix my path variables, something I had done long ago in the Python period, and overlooked in recent thrashing. In setting path and system variables in the last stage, I came to believe that I had installed GDAL twice, though there was a small difference. There was no gdalplugins files it seems from step 1, they were from step 2. So I pointed the all 3 environment settings at the C:\Program Files\GDAL sections from step 2. Now when building a C++ console app (they are working), I'm manually setting include libraries for gdal, apps, gcore, port, and ogr to the c:\gdal area in step 1, and to c:\gdal\gdal_i.lib from step 1. It seems bizarre what I am doing, pointing to both area 1 (libs) and area 2 (paths), but things for now are working. Can someone please share some "best practices" insight for setup for the benefit of this novice (me), so I hopefully can clean up the messy stuff I have created and that I am doing? I'm pretty sure what I am doing is screwy and far from best practices! ___ gdal-dev mailing list gdal-dev@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] Question on how to open a raster in HDFS using GDAL
Hello,
I am the author of the vsihdfs code, I am ready and willing to help.
I just rebuilt it from current master and was able to successfully open an
dataset via an HDFS URI with the GDAL Python bindings. I have a few
suggestions.
First, please try putting the file into a local directory and try something
like `gdalinfo /vsihdfs/file:/tmp/kahoolawe.tif` to establish a baseline.
Second, if you are using the Python bindings, please make sure that they
have been built and installed (and that you are using the ones that you
built rather than other ones that exist on your system). Instructions for
building the Python bindings can be found here:
https://trac.osgeo.org/gdal/wiki/BuildingOnUnix .
In my case, after building and installing the library and bindings, I was
able to successfully open a dataset by starting a python REPL like this:
```bash
export
LD_LIBRARY_PATH=$HOME/local/hadoop-2.7.7/lib/native:/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/amd64/server:$HOME/local/gdal-master-vsihdfs/lib
export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
export CLASSPATH=$($HOME/local/hadoop-2.7.7/bin/hadoop classpath --glob)
PYTHONPATH=$HOME/local/gdal-master-vsihdfs/lib/python2.7/site-packages
python
```
then typing this into it:
```python
from osgeo import gdal, gdalconst
ds = gdal.Open('/vsihdfs/file:/tmp/testfile.tif', gdalconst.GA_ReadOnly)
```
(I do not have easy access to and HDFS cluster right at the moment, so I
only tested a local HDFS URI.)
A note: After having done a build without HDFS support in the tree, I had
do a `make clean distclean` before I was able to get a build with working
HDFS support.
Sincerely,
James McClain
On Thu, Nov 22, 2018 at 8:13 PM Nikos Alexandris
wrote:
> * ZAZHIL-HA HERENA [2018-11-22 22:35:32 +]:
>
> >Hello, I am not sure if I should use this mailing list to ask questions
> but I wanted to try, I am a developer trying to use GDAL to open rasters in
> HDFS.
> >
> >
> >I read in GDAL documentation that starting 2.4 it is possible to open a
> raster in HDFS. I downloaded and compiled the latest source code available
> version and the generated libraries show it is 2.4 (libgdal.so.20.4.2). I
> compiled with option "-with-hdfs=yes" and "--with-java=yes".
> >
> >I am trying to open a raster using:
> >
> >
> >
> >Dataset raster = gdal.Open("/vsihdfs/hdfs://node:8020/user/hdfs
> /spatial_raster/input_raster/kahoolawe.tif", gdalconst.GA_ReadOnly);
>
> Is your path correct? There is a space here (in "/hfds /").
>
> Nikos
>
> >
> >
> >but I am getting the following error: "ERROR 4: No such file or directory"
>
> [rest deleted]
> ___
> gdal-dev mailing list
> gdal-dev@lists.osgeo.org
> https://lists.osgeo.org/mailman/listinfo/gdal-dev
--
"I prayed for freedom for twenty years, but received no answer until I
prayed with my legs."
-- Frederick Douglass
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] Question on how to open a raster in HDFS using GDAL
* ZAZHIL-HA HERENA [2018-11-22 22:35:32 +]:
Hello, I am not sure if I should use this mailing list to ask questions but I
wanted to try, I am a developer trying to use GDAL to open rasters in HDFS.
I read in GDAL documentation that starting 2.4 it is possible to open a raster in HDFS. I
downloaded and compiled the latest source code available version and the generated libraries show
it is 2.4 (libgdal.so.20.4.2). I compiled with option "-with-hdfs=yes" and
"--with-java=yes".
I am trying to open a raster using:
Dataset raster = gdal.Open("/vsihdfs/hdfs://node:8020/user/hdfs
/spatial_raster/input_raster/kahoolawe.tif", gdalconst.GA_ReadOnly);
Is your path correct? There is a space here (in "/hfds /").
Nikos
but I am getting the following error: "ERROR 4: No such file or directory"
[rest deleted]
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev
[gdal-dev] Question on how to open a raster in HDFS using GDAL
Hello, I am not sure if I should use this mailing list to ask questions but I
wanted to try, I am a developer trying to use GDAL to open rasters in HDFS.
I read in GDAL documentation that starting 2.4 it is possible to open a raster
in HDFS. I downloaded and compiled the latest source code available version and
the generated libraries show it is 2.4 (libgdal.so.20.4.2). I compiled with
option "-with-hdfs=yes" and "--with-java=yes".
I am trying to open a raster using:
Dataset raster = gdal.Open("/vsihdfs/hdfs://node:8020/user/hdfs
/spatial_raster/input_raster/kahoolawe.tif", gdalconst.GA_ReadOnly);
but I am getting the following error: "ERROR 4: No such file or directory"
HDFS system is accesible and the raster exists in that path.
could anyone please tell me if hdfs virtual system not supported yet?, or maybe
I configured it wrong when compiling?
I truly appreciate your help, thanks!
Zazhil-ha
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] problem with loading libraries
I figured out a solution to the problem. The library is loaded by the postgres process, so it needs to have its library path appended. The simplest way is to start it with command LD_LIBRARY_PATH= pg_ctl -D database start Ari Ari Jolma kirjoitti 22.11.2018 klo 17.06: Sorry, this is not strictly a GDAL question, but it is related. I have built postgis with raster support, which required gdal. Now, when I CREATE EXTENSION postgis; I get ERROR: could not load library "/home/ajolma/ods/server/parts/postgresql/lib/postgis-2.5.so": libgeos_c.so.1: cannot open shared object file: No such file or directory However, postgis-2.5.so finds libgeos: ldd parts/postgresql/lib/postgis-2.5.so does not show anything missing -- but it also does not show libgdal. I have paths to gdal and other libs in LD_LIBRARY_PATH. I don't want to make the libraries known to run-time linker with ldconfig since that would make them global(?) and this system should be strictly non-global in the system (there are several database servers etc running). Any ideas what could be the issue? Ari ___ gdal-dev mailing list gdal-dev@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] Core dumped: last GDAL dev version
Thanks Even, and sorry for the noise. The 'make clean' did the job. Chiara - Original Message - > From: "Even Rouault" > To: gdal-dev@lists.osgeo.org > Cc: "Chiara Marmo" > Sent: Thursday, November 22, 2018 3:55:18 PM > Subject: Re: [gdal-dev] Core dumped: last GDAL dev version > > On jeudi 22 novembre 2018 14:43:00 CET Chiara Marmo wrote: > > Hello, > > > > I have sinchronized my dev version of GDAL with the last upstream/master > > today (commit 9f0bbc321e11d7f88ee59af5ccdfb58204ec04f6), and experimented a > > core dumped error. > > Make sure to 'make clean' before rebuilding. > Also check if you don't have GDAL plugins corresponding to an older version > of > GDAL. > > > -- > Spatialys - Geospatial professional services > http://www.spatialys.com > -- Chiara Marmo Ingénieur de Recherche GEOPS - Paris Sud-11 Bât 509 Tel: +33 (0)1 69 15 49 03 ___ gdal-dev mailing list gdal-dev@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/gdal-dev
[gdal-dev] problem with loading libraries
Sorry, this is not strictly a GDAL question, but it is related. I have built postgis with raster support, which required gdal. Now, when I CREATE EXTENSION postgis; I get ERROR: could not load library "/home/ajolma/ods/server/parts/postgresql/lib/postgis-2.5.so": libgeos_c.so.1: cannot open shared object file: No such file or directory However, postgis-2.5.so finds libgeos: ldd parts/postgresql/lib/postgis-2.5.so does not show anything missing -- but it also does not show libgdal. I have paths to gdal and other libs in LD_LIBRARY_PATH. I don't want to make the libraries known to run-time linker with ldconfig since that would make them global(?) and this system should be strictly non-global in the system (there are several database servers etc running). Any ideas what could be the issue? Ari ___ gdal-dev mailing list gdal-dev@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] Core dumped: last GDAL dev version
On jeudi 22 novembre 2018 14:43:00 CET Chiara Marmo wrote: > Hello, > > I have sinchronized my dev version of GDAL with the last upstream/master > today (commit 9f0bbc321e11d7f88ee59af5ccdfb58204ec04f6), and experimented a > core dumped error. Make sure to 'make clean' before rebuilding. Also check if you don't have GDAL plugins corresponding to an older version of GDAL. -- Spatialys - Geospatial professional services http://www.spatialys.com ___ gdal-dev mailing list gdal-dev@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/gdal-dev
[gdal-dev] Core dumped: last GDAL dev version
Hello, I have sinchronized my dev version of GDAL with the last upstream/master today (commit 9f0bbc321e11d7f88ee59af5ccdfb58204ec04f6), and experimented a core dumped error. The following checks had been done on the master branch (not modified by me). Compilation went well, then $ ~/software/gdal/gdal/apps/ogr2ogr --version throws lt-ogr2ogr: malloc.c:2401: sysmalloc: Assertion `(old_top == initial_top (av) && old_size == 0) || ((unsigned long) (old_size) >= MINSIZE && prev_inuse (old_top) && ((unsigned long) old_end & (pagesize - 1)) == 0)' failed. Aborted (core dumped) Another example: $ ~/software/gdal/gdal/apps/gdalinfo Mercury_MESSENGER_MDIS_Basemap_BDR_Mosaic_Global_166m.tif lt-gdalinfo: malloc.c:2401: sysmalloc: Assertion `(old_top == initial_top (av) && old_size == 0) || ((unsigned long) (old_size) >= MINSIZE && prev_inuse (old_top) && ((unsigned long) old_end & (pagesize - 1)) == 0)' failed. Aborted (core dumped) This is true for all the executables in apps. The installed version of gdal apps is still working $ ogr2ogr --version GDAL 2.4.0dev-d88e13e529-dirty, released 2018/11/05 $ gdalinfo Mercury_MESSENGER_MDIS_Basemap_BDR_Mosaic_Global_166m.tif Driver: GTiff/GeoTIFF Files: Mercury_MESSENGER_MDIS_Basemap_BDR_Mosaic_Global_166m.tif Size is 92160, 46080 Coordinate System is: blahblah NoData Value=0 and it corresponds to that commit (https://github.com/epn-vespa/gdal/commit/d88e13e529928bfc72b6283b5e360c441bf40754) Where is my problem? I'm lost... Thanks for any help, Chiara -- Chiara Marmo Ingénieur de Recherche GEOPS - Paris Sud-11 Bât 509 Tel: +33 (0)1 69 15 49 03 ___ gdal-dev mailing list gdal-dev@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/gdal-dev
