Hello Felipe,
from a brief look, this code seems to be fine. Note that in the recent
version of Arrow, HdfsClient was merged into HadoopFileSystem and thus
the code will slightly differ there (should be only naming changes for
you). You should update to avoid running into any difficulties because
of that. With a backtrace or the position of the actual segfault, we
should be able to help you better.
Uwe
On Mon, Oct 30, 2017, at 07:49 AM, Felipe Aramburu wrote:
> I tried making some simple changes to the reader-writer.cc example to
> test
> it working with hdfs. I am pretty sure I have set up my hdfs properly
> because I can do things like create directories. I copied over the
> parquet
> file generate by the test to hdfs and tried to load it in as follows:
>
>
>
> /**********************************************************************************
> PARQUET READER EXAMPLE
>
> **********************************************************************************/
>
> try {
> // Create a ParquetReader instance
>
> std::shared_ptr<arrow::io::HdfsClient> hdfs;
> arrow::Status connectionStat =
> arrow::io::HdfsClient::Connect(&hdfsConfig,
> &hdfs);
>
> hdfs->MakeDirectory("/tester");
> std::cout<<"made directory"<<std::endl;
> std::shared_ptr<arrow::io::HdfsReadableFile> in_file;
> arrow::Status stat =
> hdfs->OpenReadable("/testing2/test2.parquet",1024*1024,&in_file);
> std::cout<<"opened readable"<<std::endl;
> std::shared_ptr<parquet::FileMetaData> file_metadata;
> std::unique_ptr<parquet::ParquetFileReader> parquet_reader =
>
>
> parquet::ParquetFileReader::Open(in_file,parquet::default_reader_properties(),file_metadata);
>
>
> Is this the preferred way of doing this? I should be clear its
> segfaulting
> but due to a linker error that I am not able to debug the parquet-cpp
> files at the moment (will fix anc update with more info as soon as I
> can).
> I did want to make sure that this is the right way of doing this. I had
> trouble finding documentation on this.
>
> Felipe
> ᐧ
