Thank you for the reply.
Nikos,
The path is correct, the space was an error in my redaction.
James,
I tried to run a GDALINFO in my Linux command line using the GDAL I configured
and installed and I am not able to open the raster, even trying to read it from
local:
-bash-4.2$ gdalinfo
/vsihdfs/file:/scratch/zherena/test/write/data/rasters/hawaii.tif
ERROR 4:
/vsihdfs/file:/scratch/zherena/test/write/data/rasters/hawaii.tif: No such file
or directory
gdalinfo failed - unable to open
'/vsihdfs/file:/scratch/zherena/test/write/data/rasters/hawaii.tif'.
If I try the same commad withouth /vsihdfs/file:/... it works fine.
I also tried:
-bash-4.2$ gdalinfo --version
GDAL 2.3.2, released 2018/09/21
Version says 2.3.2 but libraries say: libgdal.so.20.4.2 . I am not sure if I
got the latest code, this is the first time I compile it myself, I used this
link to download source code:
http://download.osgeo.org/gdal/CURRENT/gdal-2.3.2.tar.gz
And I have a doubt, when configuring I used: --with-hdfs=yes, made another
using: --with-hdfs = /scratch/zherena/gdal/hadoop/lib/native. Both tries did
not work. I am not really sure what should I pass as a parameter to --with-hdfs?
This is the list of steps I used to build, I am using Java in my application
where I plan to use gdal:
* make clean distclean(per your suggestion)
* ./configure --prefix=/scratch/zherena/gdal/build/gdal-2.3.2/outputb/
--with-complete=yes --with-java=yes --with-swig-java=yes
--with-hdfs=/scratch/zherena/gdal/hadoop/lib/native
--with-curl=/usr/bin/curl-config
* make
* make install
* cd /swig/java
* make veryclean
* make
* cp *.so ../../outputb/lib/
* cp *.jar ../../outputb/lib/
Maybe I am missing something during compile?
Thank you!!
Zazhil-ha
________________________________
From: gdal-dev <gdal-dev-boun...@lists.osgeo.org> on behalf of James McClain
<james.mccl...@gmail.com>
Sent: Thursday, November 22, 2018 8:27 PM
Cc: gdal-dev@lists.osgeo.org
Subject: Re: [gdal-dev] Question on how to open a raster in HDFS using GDAL
Hello,
I am the author of the vsihdfs code, I am ready and willing to help.
I just rebuilt it from current master and was able to successfully open an
dataset via an HDFS URI with the GDAL Python bindings. I have a few
suggestions.
First, please try putting the file into a local directory and try something
like `gdalinfo /vsihdfs/file:/tmp/kahoolawe.tif` to establish a baseline.
Second, if you are using the Python bindings, please make sure that they have
been built and installed (and that you are using the ones that you built rather
than other ones that exist on your system). Instructions for building the
Python bindings can be found here:
https://trac.osgeo.org/gdal/wiki/BuildingOnUnix .
In my case, after building and installing the library and bindings, I was able
to successfully open a dataset by starting a python REPL like this:
```bash
export
LD_LIBRARY_PATH=$HOME/local/hadoop-2.7.7/lib/native:/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/amd64/server:$HOME/local/gdal-master-vsihdfs/lib
export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
export CLASSPATH=$($HOME/local/hadoop-2.7.7/bin/hadoop classpath --glob)
PYTHONPATH=$HOME/local/gdal-master-vsihdfs/lib/python2.7/site-packages python
```
then typing this into it:
```python
from osgeo import gdal, gdalconst
ds = gdal.Open('/vsihdfs/file:/tmp/testfile.tif', gdalconst.GA_ReadOnly)
```
(I do not have easy access to and HDFS cluster right at the moment, so I only
tested a local HDFS URI.)
A note: After having done a build without HDFS support in the tree, I had do a
`make clean distclean` before I was able to get a build with working HDFS
support.
Sincerely,
James McClain
On Thu, Nov 22, 2018 at 8:13 PM Nikos Alexandris
<n...@nikosalexandris.net<mailto:n...@nikosalexandris.net>> wrote:
* ZAZHIL-HA HERENA <zazhil...@hotmail.com<mailto:zazhil...@hotmail.com>>
[2018-11-22 22:35:32 +0000]:
>Hello, I am not sure if I should use this mailing list to ask questions but I
>wanted to try, I am a developer trying to use GDAL to open rasters in HDFS.
>
>
>I read in GDAL documentation that starting 2.4 it is possible to open a raster
>in HDFS. I downloaded and compiled the latest source code available version
>and the generated libraries show it is 2.4 (libgdal.so.20.4.2). I compiled
>with option "-with-hdfs=yes" and "--with-java=yes".
>
>I am trying to open a raster using:
>
>
>
> Dataset raster = gdal.Open("/vsihdfs/hdfs://node:8020/user/hdfs
> /spatial_raster/input_raster/kahoolawe.tif", gdalconst.GA_ReadOnly);
Is your path correct? There is a space here (in "/hfds /").
Nikos
>
>
>but I am getting the following error: "ERROR 4: No such file or directory"
[rest deleted]
_______________________________________________
gdal-dev mailing list
gdal-dev@lists.osgeo.org<mailto:gdal-dev@lists.osgeo.org>
https://lists.osgeo.org/mailman/listinfo/gdal-dev
--
"I prayed for freedom for twenty years, but received no answer until I prayed
with my legs."
-- Frederick Douglass
_______________________________________________
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev
